{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Holiday Package Prediciton\n",
    "\n",
    "### 1) Problem statement.\n",
    "\"Trips & Travel.Com\" company wants to enable and establish a viable business model to expand the customer base.\n",
    "One of the ways to expand the customer base is to introduce a new offering of packages. Currently, there are 5 types of packages the company is offering * Basic, Standard, Deluxe, Super Deluxe, King. Looking at the data of the last year, we observed that 18% of the customers purchased the packages. However, the marketing cost was quite high because customers were contacted at random without looking at the available information.\n",
    "The company is now planning to launch a new product i.e. Wellness Tourism Package. Wellness Tourism is defined as Travel that allows the traveler to maintain, enhance or kick-start a healthy lifestyle, and support or increase one's sense of well-being.\n",
    "However, this time company wants to harness the available data of existing and potential customers to make the marketing expenditure more efficient.\n",
    "### 2) Data Collection.\n",
    "The Dataset is collected from https://www.kaggle.com/datasets/susant4learning/holiday-package-purchase-prediction\n",
    "The data consists of 20 column and 4888 rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "## importing important libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import plotly.express as px\n",
    "import warnings\n",
    "\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CustomerID</th>\n",
       "      <th>ProdTaken</th>\n",
       "      <th>Age</th>\n",
       "      <th>TypeofContact</th>\n",
       "      <th>CityTier</th>\n",
       "      <th>DurationOfPitch</th>\n",
       "      <th>Occupation</th>\n",
       "      <th>Gender</th>\n",
       "      <th>NumberOfPersonVisiting</th>\n",
       "      <th>NumberOfFollowups</th>\n",
       "      <th>ProductPitched</th>\n",
       "      <th>PreferredPropertyStar</th>\n",
       "      <th>MaritalStatus</th>\n",
       "      <th>NumberOfTrips</th>\n",
       "      <th>Passport</th>\n",
       "      <th>PitchSatisfactionScore</th>\n",
       "      <th>OwnCar</th>\n",
       "      <th>NumberOfChildrenVisiting</th>\n",
       "      <th>Designation</th>\n",
       "      <th>MonthlyIncome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>200000</td>\n",
       "      <td>1</td>\n",
       "      <td>41.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>3</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>3</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Single</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20993.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>200001</td>\n",
       "      <td>0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>14.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Male</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20130.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>200002</td>\n",
       "      <td>1</td>\n",
       "      <td>37.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Free Lancer</td>\n",
       "      <td>Male</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Single</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17090.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>200003</td>\n",
       "      <td>0</td>\n",
       "      <td>33.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17909.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>200004</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Small Business</td>\n",
       "      <td>Male</td>\n",
       "      <td>2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>18468.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   CustomerID  ProdTaken   Age    TypeofContact  CityTier  DurationOfPitch  \\\n",
       "0      200000          1  41.0     Self Enquiry         3              6.0   \n",
       "1      200001          0  49.0  Company Invited         1             14.0   \n",
       "2      200002          1  37.0     Self Enquiry         1              8.0   \n",
       "3      200003          0  33.0  Company Invited         1              9.0   \n",
       "4      200004          0   NaN     Self Enquiry         1              8.0   \n",
       "\n",
       "       Occupation  Gender  NumberOfPersonVisiting  NumberOfFollowups  \\\n",
       "0        Salaried  Female                       3                3.0   \n",
       "1        Salaried    Male                       3                4.0   \n",
       "2     Free Lancer    Male                       3                4.0   \n",
       "3        Salaried  Female                       2                3.0   \n",
       "4  Small Business    Male                       2                3.0   \n",
       "\n",
       "  ProductPitched  PreferredPropertyStar MaritalStatus  NumberOfTrips  \\\n",
       "0         Deluxe                    3.0        Single            1.0   \n",
       "1         Deluxe                    4.0      Divorced            2.0   \n",
       "2          Basic                    3.0        Single            7.0   \n",
       "3          Basic                    3.0      Divorced            2.0   \n",
       "4          Basic                    4.0      Divorced            1.0   \n",
       "\n",
       "   Passport  PitchSatisfactionScore  OwnCar  NumberOfChildrenVisiting  \\\n",
       "0         1                       2       1                       0.0   \n",
       "1         0                       3       1                       2.0   \n",
       "2         1                       3       0                       0.0   \n",
       "3         1                       5       1                       1.0   \n",
       "4         0                       5       1                       0.0   \n",
       "\n",
       "  Designation  MonthlyIncome  \n",
       "0     Manager        20993.0  \n",
       "1     Manager        20130.0  \n",
       "2   Executive        17090.0  \n",
       "3   Executive        17909.0  \n",
       "4   Executive        18468.0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"Travel.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Cleaning\n",
    "### Handling Missing values\n",
    "1. Handling Missing values\n",
    "2. Handling Duplicates\n",
    "3. Check data type\n",
    "4. Understand the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CustomerID                    0\n",
       "ProdTaken                     0\n",
       "Age                         226\n",
       "TypeofContact                25\n",
       "CityTier                      0\n",
       "DurationOfPitch             251\n",
       "Occupation                    0\n",
       "Gender                        0\n",
       "NumberOfPersonVisiting        0\n",
       "NumberOfFollowups            45\n",
       "ProductPitched                0\n",
       "PreferredPropertyStar        26\n",
       "MaritalStatus                 0\n",
       "NumberOfTrips               140\n",
       "Passport                      0\n",
       "PitchSatisfactionScore        0\n",
       "OwnCar                        0\n",
       "NumberOfChildrenVisiting     66\n",
       "Designation                   0\n",
       "MonthlyIncome               233\n",
       "dtype: int64"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Male       2916\n",
       "Female     1817\n",
       "Fe Male     155\n",
       "Name: Gender, dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "### Check all the categories \n",
    "df['Gender'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Married      2340\n",
       "Divorced      950\n",
       "Single        916\n",
       "Unmarried     682\n",
       "Name: MaritalStatus, dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['MaritalStatus'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Self Enquiry       3444\n",
       "Company Invited    1419\n",
       "Name: TypeofContact, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['TypeofContact'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['Gender'] = df['Gender'].replace('Fe Male', 'Female')\n",
    "df['MaritalStatus'] = df['MaritalStatus'].replace('Single', 'Unmarried')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Male      2916\n",
       "Female    1972\n",
       "Name: Gender, dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "### Check all the categories \n",
    "df['Gender'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CustomerID</th>\n",
       "      <th>ProdTaken</th>\n",
       "      <th>Age</th>\n",
       "      <th>TypeofContact</th>\n",
       "      <th>CityTier</th>\n",
       "      <th>DurationOfPitch</th>\n",
       "      <th>Occupation</th>\n",
       "      <th>Gender</th>\n",
       "      <th>NumberOfPersonVisiting</th>\n",
       "      <th>NumberOfFollowups</th>\n",
       "      <th>ProductPitched</th>\n",
       "      <th>PreferredPropertyStar</th>\n",
       "      <th>MaritalStatus</th>\n",
       "      <th>NumberOfTrips</th>\n",
       "      <th>Passport</th>\n",
       "      <th>PitchSatisfactionScore</th>\n",
       "      <th>OwnCar</th>\n",
       "      <th>NumberOfChildrenVisiting</th>\n",
       "      <th>Designation</th>\n",
       "      <th>MonthlyIncome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>200000</td>\n",
       "      <td>1</td>\n",
       "      <td>41.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>3</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>3</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20993.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>200001</td>\n",
       "      <td>0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>14.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Male</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20130.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>200002</td>\n",
       "      <td>1</td>\n",
       "      <td>37.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Free Lancer</td>\n",
       "      <td>Male</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17090.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>200003</td>\n",
       "      <td>0</td>\n",
       "      <td>33.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17909.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>200004</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Small Business</td>\n",
       "      <td>Male</td>\n",
       "      <td>2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>18468.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   CustomerID  ProdTaken   Age    TypeofContact  CityTier  DurationOfPitch  \\\n",
       "0      200000          1  41.0     Self Enquiry         3              6.0   \n",
       "1      200001          0  49.0  Company Invited         1             14.0   \n",
       "2      200002          1  37.0     Self Enquiry         1              8.0   \n",
       "3      200003          0  33.0  Company Invited         1              9.0   \n",
       "4      200004          0   NaN     Self Enquiry         1              8.0   \n",
       "\n",
       "       Occupation  Gender  NumberOfPersonVisiting  NumberOfFollowups  \\\n",
       "0        Salaried  Female                       3                3.0   \n",
       "1        Salaried    Male                       3                4.0   \n",
       "2     Free Lancer    Male                       3                4.0   \n",
       "3        Salaried  Female                       2                3.0   \n",
       "4  Small Business    Male                       2                3.0   \n",
       "\n",
       "  ProductPitched  PreferredPropertyStar MaritalStatus  NumberOfTrips  \\\n",
       "0         Deluxe                    3.0     Unmarried            1.0   \n",
       "1         Deluxe                    4.0      Divorced            2.0   \n",
       "2          Basic                    3.0     Unmarried            7.0   \n",
       "3          Basic                    3.0      Divorced            2.0   \n",
       "4          Basic                    4.0      Divorced            1.0   \n",
       "\n",
       "   Passport  PitchSatisfactionScore  OwnCar  NumberOfChildrenVisiting  \\\n",
       "0         1                       2       1                       0.0   \n",
       "1         0                       3       1                       2.0   \n",
       "2         1                       3       0                       0.0   \n",
       "3         1                       5       1                       1.0   \n",
       "4         0                       5       1                       0.0   \n",
       "\n",
       "  Designation  MonthlyIncome  \n",
       "0     Manager        20993.0  \n",
       "1     Manager        20130.0  \n",
       "2   Executive        17090.0  \n",
       "3   Executive        17909.0  \n",
       "4   Executive        18468.0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Age 4.62357 % missing values\n",
      "TypeofContact 0.51146 % missing values\n",
      "DurationOfPitch 5.13502 % missing values\n",
      "NumberOfFollowups 0.92062 % missing values\n",
      "PreferredPropertyStar 0.53191 % missing values\n",
      "NumberOfTrips 2.86416 % missing values\n",
      "NumberOfChildrenVisiting 1.35025 % missing values\n",
      "MonthlyIncome 4.76678 % missing values\n"
     ]
    }
   ],
   "source": [
    "## Check Misssing Values\n",
    "##these are the features with nan value\n",
    "features_with_na=[features for features in df.columns if df[features].isnull().sum()>=1]\n",
    "for feature in features_with_na:\n",
    "    print(feature,np.round(df[feature].isnull().mean()*100,5), '% missing values')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>DurationOfPitch</th>\n",
       "      <th>NumberOfFollowups</th>\n",
       "      <th>PreferredPropertyStar</th>\n",
       "      <th>NumberOfTrips</th>\n",
       "      <th>NumberOfChildrenVisiting</th>\n",
       "      <th>MonthlyIncome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>4662.000000</td>\n",
       "      <td>4637.000000</td>\n",
       "      <td>4843.000000</td>\n",
       "      <td>4862.000000</td>\n",
       "      <td>4748.000000</td>\n",
       "      <td>4822.000000</td>\n",
       "      <td>4655.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>37.622265</td>\n",
       "      <td>15.490835</td>\n",
       "      <td>3.708445</td>\n",
       "      <td>3.581037</td>\n",
       "      <td>3.236521</td>\n",
       "      <td>1.187267</td>\n",
       "      <td>23619.853491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>9.316387</td>\n",
       "      <td>8.519643</td>\n",
       "      <td>1.002509</td>\n",
       "      <td>0.798009</td>\n",
       "      <td>1.849019</td>\n",
       "      <td>0.857861</td>\n",
       "      <td>5380.698361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>18.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>31.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>20346.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>36.000000</td>\n",
       "      <td>13.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>22347.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>44.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>25571.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>61.000000</td>\n",
       "      <td>127.000000</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>98678.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Age  DurationOfPitch  NumberOfFollowups  PreferredPropertyStar  \\\n",
       "count  4662.000000      4637.000000        4843.000000            4862.000000   \n",
       "mean     37.622265        15.490835           3.708445               3.581037   \n",
       "std       9.316387         8.519643           1.002509               0.798009   \n",
       "min      18.000000         5.000000           1.000000               3.000000   \n",
       "25%      31.000000         9.000000           3.000000               3.000000   \n",
       "50%      36.000000        13.000000           4.000000               3.000000   \n",
       "75%      44.000000        20.000000           4.000000               4.000000   \n",
       "max      61.000000       127.000000           6.000000               5.000000   \n",
       "\n",
       "       NumberOfTrips  NumberOfChildrenVisiting  MonthlyIncome  \n",
       "count    4748.000000               4822.000000    4655.000000  \n",
       "mean        3.236521                  1.187267   23619.853491  \n",
       "std         1.849019                  0.857861    5380.698361  \n",
       "min         1.000000                  0.000000    1000.000000  \n",
       "25%         2.000000                  1.000000   20346.000000  \n",
       "50%         3.000000                  1.000000   22347.000000  \n",
       "75%         4.000000                  2.000000   25571.000000  \n",
       "max        22.000000                  3.000000   98678.000000  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# statistics on numerical columns (Null cols)\n",
    "df[features_with_na].select_dtypes(exclude='object').describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imputing Null values\n",
    "1. Impute Median value for Age column\n",
    "2. Impute Mode for Type of Contract\n",
    "3. Impute Median for Duration of Pitch\n",
    "4. Impute Mode for NumberofFollowup as it is Discrete feature\n",
    "5. Impute Mode for PreferredPropertyStar\n",
    "6. Impute Median for NumberofTrips\n",
    "7. Impute Mode for NumberOfChildrenVisiting\n",
    "8. Impute Median for MonthlyIncome"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Age\n",
    "df.Age.fillna(df.Age.median(), inplace=True)\n",
    "\n",
    "#TypeofContract\n",
    "df.TypeofContact.fillna(df.TypeofContact.mode()[0], inplace=True)\n",
    "\n",
    "#DurationOfPitch\n",
    "df.DurationOfPitch.fillna(df.DurationOfPitch.median(), inplace=True)\n",
    "\n",
    "#NumberOfFollowups\n",
    "df.NumberOfFollowups.fillna(df.NumberOfFollowups.mode()[0], inplace=True)\n",
    "\n",
    "#PreferredPropertyStar\n",
    "df.PreferredPropertyStar.fillna(df.PreferredPropertyStar.mode()[0], inplace=True)\n",
    "\n",
    "#NumberOfTrips\n",
    "df.NumberOfTrips.fillna(df.NumberOfTrips.median(), inplace=True)\n",
    "\n",
    "#NumberOfChildrenVisiting\n",
    "df.NumberOfChildrenVisiting.fillna(df.NumberOfChildrenVisiting.mode()[0], inplace=True)\n",
    "\n",
    "#MonthlyIncome\n",
    "df.MonthlyIncome.fillna(df.MonthlyIncome.median(), inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CustomerID                  0\n",
       "ProdTaken                   0\n",
       "Age                         0\n",
       "TypeofContact               0\n",
       "CityTier                    0\n",
       "DurationOfPitch             0\n",
       "Occupation                  0\n",
       "Gender                      0\n",
       "NumberOfPersonVisiting      0\n",
       "NumberOfFollowups           0\n",
       "ProductPitched              0\n",
       "PreferredPropertyStar       0\n",
       "MaritalStatus               0\n",
       "NumberOfTrips               0\n",
       "Passport                    0\n",
       "PitchSatisfactionScore      0\n",
       "OwnCar                      0\n",
       "NumberOfChildrenVisiting    0\n",
       "Designation                 0\n",
       "MonthlyIncome               0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()\n",
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.drop('CustomerID', inplace=True, axis=1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Engineering\n",
    "\n",
    "### Feature Extraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ProdTaken</th>\n",
       "      <th>Age</th>\n",
       "      <th>TypeofContact</th>\n",
       "      <th>CityTier</th>\n",
       "      <th>DurationOfPitch</th>\n",
       "      <th>Occupation</th>\n",
       "      <th>Gender</th>\n",
       "      <th>NumberOfPersonVisiting</th>\n",
       "      <th>NumberOfFollowups</th>\n",
       "      <th>ProductPitched</th>\n",
       "      <th>PreferredPropertyStar</th>\n",
       "      <th>MaritalStatus</th>\n",
       "      <th>NumberOfTrips</th>\n",
       "      <th>Passport</th>\n",
       "      <th>PitchSatisfactionScore</th>\n",
       "      <th>OwnCar</th>\n",
       "      <th>NumberOfChildrenVisiting</th>\n",
       "      <th>Designation</th>\n",
       "      <th>MonthlyIncome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>41.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>3</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>3</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20993.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>14.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Male</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20130.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>37.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Free Lancer</td>\n",
       "      <td>Male</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17090.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>33.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17909.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>36.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Small Business</td>\n",
       "      <td>Male</td>\n",
       "      <td>2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>18468.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ProdTaken   Age    TypeofContact  CityTier  DurationOfPitch  \\\n",
       "0          1  41.0     Self Enquiry         3              6.0   \n",
       "1          0  49.0  Company Invited         1             14.0   \n",
       "2          1  37.0     Self Enquiry         1              8.0   \n",
       "3          0  33.0  Company Invited         1              9.0   \n",
       "4          0  36.0     Self Enquiry         1              8.0   \n",
       "\n",
       "       Occupation  Gender  NumberOfPersonVisiting  NumberOfFollowups  \\\n",
       "0        Salaried  Female                       3                3.0   \n",
       "1        Salaried    Male                       3                4.0   \n",
       "2     Free Lancer    Male                       3                4.0   \n",
       "3        Salaried  Female                       2                3.0   \n",
       "4  Small Business    Male                       2                3.0   \n",
       "\n",
       "  ProductPitched  PreferredPropertyStar MaritalStatus  NumberOfTrips  \\\n",
       "0         Deluxe                    3.0     Unmarried            1.0   \n",
       "1         Deluxe                    4.0      Divorced            2.0   \n",
       "2          Basic                    3.0     Unmarried            7.0   \n",
       "3          Basic                    3.0      Divorced            2.0   \n",
       "4          Basic                    4.0      Divorced            1.0   \n",
       "\n",
       "   Passport  PitchSatisfactionScore  OwnCar  NumberOfChildrenVisiting  \\\n",
       "0         1                       2       1                       0.0   \n",
       "1         0                       3       1                       2.0   \n",
       "2         1                       3       0                       0.0   \n",
       "3         1                       5       1                       1.0   \n",
       "4         0                       5       1                       0.0   \n",
       "\n",
       "  Designation  MonthlyIncome  \n",
       "0     Manager        20993.0  \n",
       "1     Manager        20130.0  \n",
       "2   Executive        17090.0  \n",
       "3   Executive        17909.0  \n",
       "4   Executive        18468.0  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create new column for feature\n",
    "df['TotalVisiting'] = df['NumberOfPersonVisiting'] + df['NumberOfChildrenVisiting']\n",
    "df.drop(columns=['NumberOfPersonVisiting', 'NumberOfChildrenVisiting'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Num of Numerical Features : 12\n"
     ]
    }
   ],
   "source": [
    "## get all the numeric features\n",
    "num_features = [feature for feature in df.columns if df[feature].dtype != 'O']\n",
    "print('Num of Numerical Features :', len(num_features))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Num of Categorical Features : 6\n"
     ]
    }
   ],
   "source": [
    "##categorical features\n",
    "cat_features = [feature for feature in df.columns if df[feature].dtype == 'O']\n",
    "print('Num of Categorical Features :', len(cat_features))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Num of Discrete Features : 9\n"
     ]
    }
   ],
   "source": [
    "## Discrete features\n",
    "discrete_features=[feature for feature in num_features if len(df[feature].unique())<=25]\n",
    "print('Num of Discrete Features :',len(discrete_features))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Num of Continuous Features : 3\n"
     ]
    }
   ],
   "source": [
    "## coontinuous features\n",
    "continuous_features=[feature for feature in num_features if feature not in discrete_features]\n",
    "print('Num of Continuous Features :',len(continuous_features))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ProdTaken</th>\n",
       "      <th>Age</th>\n",
       "      <th>TypeofContact</th>\n",
       "      <th>CityTier</th>\n",
       "      <th>DurationOfPitch</th>\n",
       "      <th>Occupation</th>\n",
       "      <th>Gender</th>\n",
       "      <th>NumberOfFollowups</th>\n",
       "      <th>ProductPitched</th>\n",
       "      <th>PreferredPropertyStar</th>\n",
       "      <th>MaritalStatus</th>\n",
       "      <th>NumberOfTrips</th>\n",
       "      <th>Passport</th>\n",
       "      <th>PitchSatisfactionScore</th>\n",
       "      <th>OwnCar</th>\n",
       "      <th>Designation</th>\n",
       "      <th>MonthlyIncome</th>\n",
       "      <th>TotalVisiting</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>41.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>3</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20993.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>14.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Male</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20130.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>37.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Free Lancer</td>\n",
       "      <td>Male</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17090.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>33.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17909.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>36.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Small Business</td>\n",
       "      <td>Male</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>Executive</td>\n",
       "      <td>18468.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ProdTaken   Age    TypeofContact  CityTier  DurationOfPitch  \\\n",
       "0          1  41.0     Self Enquiry         3              6.0   \n",
       "1          0  49.0  Company Invited         1             14.0   \n",
       "2          1  37.0     Self Enquiry         1              8.0   \n",
       "3          0  33.0  Company Invited         1              9.0   \n",
       "4          0  36.0     Self Enquiry         1              8.0   \n",
       "\n",
       "       Occupation  Gender  NumberOfFollowups ProductPitched  \\\n",
       "0        Salaried  Female                3.0         Deluxe   \n",
       "1        Salaried    Male                4.0         Deluxe   \n",
       "2     Free Lancer    Male                4.0          Basic   \n",
       "3        Salaried  Female                3.0          Basic   \n",
       "4  Small Business    Male                3.0          Basic   \n",
       "\n",
       "   PreferredPropertyStar MaritalStatus  NumberOfTrips  Passport  \\\n",
       "0                    3.0     Unmarried            1.0         1   \n",
       "1                    4.0      Divorced            2.0         0   \n",
       "2                    3.0     Unmarried            7.0         1   \n",
       "3                    3.0      Divorced            2.0         1   \n",
       "4                    4.0      Divorced            1.0         0   \n",
       "\n",
       "   PitchSatisfactionScore  OwnCar Designation  MonthlyIncome  TotalVisiting  \n",
       "0                       2       1     Manager        20993.0            3.0  \n",
       "1                       3       1     Manager        20130.0            5.0  \n",
       "2                       3       0   Executive        17090.0            3.0  \n",
       "3                       5       1   Executive        17909.0            3.0  \n",
       "4                       5       1   Executive        18468.0            2.0  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train Test Split And Model Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X = df.drop(['ProdTaken'], axis=1)\n",
    "y = df['ProdTaken']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>TypeofContact</th>\n",
       "      <th>CityTier</th>\n",
       "      <th>DurationOfPitch</th>\n",
       "      <th>Occupation</th>\n",
       "      <th>Gender</th>\n",
       "      <th>NumberOfFollowups</th>\n",
       "      <th>ProductPitched</th>\n",
       "      <th>PreferredPropertyStar</th>\n",
       "      <th>MaritalStatus</th>\n",
       "      <th>NumberOfTrips</th>\n",
       "      <th>Passport</th>\n",
       "      <th>PitchSatisfactionScore</th>\n",
       "      <th>OwnCar</th>\n",
       "      <th>Designation</th>\n",
       "      <th>MonthlyIncome</th>\n",
       "      <th>TotalVisiting</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>41.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>3</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20993.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>49.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>14.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Male</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20130.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>37.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Free Lancer</td>\n",
       "      <td>Male</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17090.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>33.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17909.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>36.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Small Business</td>\n",
       "      <td>Male</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>Executive</td>\n",
       "      <td>18468.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Age    TypeofContact  CityTier  DurationOfPitch      Occupation  Gender  \\\n",
       "0  41.0     Self Enquiry         3              6.0        Salaried  Female   \n",
       "1  49.0  Company Invited         1             14.0        Salaried    Male   \n",
       "2  37.0     Self Enquiry         1              8.0     Free Lancer    Male   \n",
       "3  33.0  Company Invited         1              9.0        Salaried  Female   \n",
       "4  36.0     Self Enquiry         1              8.0  Small Business    Male   \n",
       "\n",
       "   NumberOfFollowups ProductPitched  PreferredPropertyStar MaritalStatus  \\\n",
       "0                3.0         Deluxe                    3.0     Unmarried   \n",
       "1                4.0         Deluxe                    4.0      Divorced   \n",
       "2                4.0          Basic                    3.0     Unmarried   \n",
       "3                3.0          Basic                    3.0      Divorced   \n",
       "4                3.0          Basic                    4.0      Divorced   \n",
       "\n",
       "   NumberOfTrips  Passport  PitchSatisfactionScore  OwnCar Designation  \\\n",
       "0            1.0         1                       2       1     Manager   \n",
       "1            2.0         0                       3       1     Manager   \n",
       "2            7.0         1                       3       0   Executive   \n",
       "3            2.0         1                       5       1   Executive   \n",
       "4            1.0         0                       5       1   Executive   \n",
       "\n",
       "   MonthlyIncome  TotalVisiting  \n",
       "0        20993.0            3.0  \n",
       "1        20130.0            5.0  \n",
       "2        17090.0            3.0  \n",
       "3        17909.0            3.0  \n",
       "4        18468.0            2.0  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3968\n",
       "1     920\n",
       "Name: ProdTaken, dtype: int64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>TypeofContact</th>\n",
       "      <th>CityTier</th>\n",
       "      <th>DurationOfPitch</th>\n",
       "      <th>Occupation</th>\n",
       "      <th>Gender</th>\n",
       "      <th>NumberOfFollowups</th>\n",
       "      <th>ProductPitched</th>\n",
       "      <th>PreferredPropertyStar</th>\n",
       "      <th>MaritalStatus</th>\n",
       "      <th>NumberOfTrips</th>\n",
       "      <th>Passport</th>\n",
       "      <th>PitchSatisfactionScore</th>\n",
       "      <th>OwnCar</th>\n",
       "      <th>Designation</th>\n",
       "      <th>MonthlyIncome</th>\n",
       "      <th>TotalVisiting</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>41.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>3</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20993.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>49.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>14.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Male</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Deluxe</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>Manager</td>\n",
       "      <td>20130.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>37.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Free Lancer</td>\n",
       "      <td>Male</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Unmarried</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17090.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>33.0</td>\n",
       "      <td>Company Invited</td>\n",
       "      <td>1</td>\n",
       "      <td>9.0</td>\n",
       "      <td>Salaried</td>\n",
       "      <td>Female</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>Executive</td>\n",
       "      <td>17909.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>36.0</td>\n",
       "      <td>Self Enquiry</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Small Business</td>\n",
       "      <td>Male</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Basic</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>Executive</td>\n",
       "      <td>18468.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Age    TypeofContact  CityTier  DurationOfPitch      Occupation  Gender  \\\n",
       "0  41.0     Self Enquiry         3              6.0        Salaried  Female   \n",
       "1  49.0  Company Invited         1             14.0        Salaried    Male   \n",
       "2  37.0     Self Enquiry         1              8.0     Free Lancer    Male   \n",
       "3  33.0  Company Invited         1              9.0        Salaried  Female   \n",
       "4  36.0     Self Enquiry         1              8.0  Small Business    Male   \n",
       "\n",
       "   NumberOfFollowups ProductPitched  PreferredPropertyStar MaritalStatus  \\\n",
       "0                3.0         Deluxe                    3.0     Unmarried   \n",
       "1                4.0         Deluxe                    4.0      Divorced   \n",
       "2                4.0          Basic                    3.0     Unmarried   \n",
       "3                3.0          Basic                    3.0      Divorced   \n",
       "4                3.0          Basic                    4.0      Divorced   \n",
       "\n",
       "   NumberOfTrips  Passport  PitchSatisfactionScore  OwnCar Designation  \\\n",
       "0            1.0         1                       2       1     Manager   \n",
       "1            2.0         0                       3       1     Manager   \n",
       "2            7.0         1                       3       0   Executive   \n",
       "3            2.0         1                       5       1   Executive   \n",
       "4            1.0         0                       5       1   Executive   \n",
       "\n",
       "   MonthlyIncome  TotalVisiting  \n",
       "0        20993.0            3.0  \n",
       "1        20130.0            5.0  \n",
       "2        17090.0            3.0  \n",
       "3        17909.0            3.0  \n",
       "4        18468.0            2.0  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((3910, 17), (978, 17))"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# separate dataset into train and test\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2,random_state=42)\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4888 entries, 0 to 4887\n",
      "Data columns (total 17 columns):\n",
      " #   Column                  Non-Null Count  Dtype  \n",
      "---  ------                  --------------  -----  \n",
      " 0   Age                     4888 non-null   float64\n",
      " 1   TypeofContact           4888 non-null   object \n",
      " 2   CityTier                4888 non-null   int64  \n",
      " 3   DurationOfPitch         4888 non-null   float64\n",
      " 4   Occupation              4888 non-null   object \n",
      " 5   Gender                  4888 non-null   object \n",
      " 6   NumberOfFollowups       4888 non-null   float64\n",
      " 7   ProductPitched          4888 non-null   object \n",
      " 8   PreferredPropertyStar   4888 non-null   float64\n",
      " 9   MaritalStatus           4888 non-null   object \n",
      " 10  NumberOfTrips           4888 non-null   float64\n",
      " 11  Passport                4888 non-null   int64  \n",
      " 12  PitchSatisfactionScore  4888 non-null   int64  \n",
      " 13  OwnCar                  4888 non-null   int64  \n",
      " 14  Designation             4888 non-null   object \n",
      " 15  MonthlyIncome           4888 non-null   float64\n",
      " 16  TotalVisiting           4888 non-null   float64\n",
      "dtypes: float64(7), int64(4), object(6)\n",
      "memory usage: 649.3+ KB\n"
     ]
    }
   ],
   "source": [
    "X.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create Column Transformer with 3 types of transformers\n",
    "cat_features = X.select_dtypes(include=\"object\").columns\n",
    "num_features = X.select_dtypes(exclude=\"object\").columns\n",
    "\n",
    "from sklearn.preprocessing import OneHotEncoder, StandardScaler\n",
    "from sklearn.compose import ColumnTransformer\n",
    "\n",
    "numeric_transformer = StandardScaler()\n",
    "oh_transformer = OneHotEncoder(drop='first')\n",
    "\n",
    "preprocessor = ColumnTransformer(\n",
    "    [\n",
    "         (\"OneHotEncoder\", oh_transformer, cat_features),\n",
    "          (\"StandardScaler\", numeric_transformer, num_features)\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ColumnTransformer(transformers=[('OneHotEncoder', OneHotEncoder(drop='first'),\n",
       "                                 Index(['TypeofContact', 'Occupation', 'Gender', 'ProductPitched',\n",
       "       'MaritalStatus', 'Designation'],\n",
       "      dtype='object')),\n",
       "                                ('StandardScaler', StandardScaler(),\n",
       "                                 Index(['Age', 'CityTier', 'DurationOfPitch', 'NumberOfFollowups',\n",
       "       'PreferredPropertyStar', 'NumberOfTrips', 'Passport',\n",
       "       'PitchSatisfactionScore', 'OwnCar', 'MonthlyIncome', 'TotalVisiting'],\n",
       "      dtype='object'))])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preprocessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "## applying Trnsformation in training(fit_transform)\n",
    "X_train=preprocessor.fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "      <th>20</th>\n",
       "      <th>21</th>\n",
       "      <th>22</th>\n",
       "      <th>23</th>\n",
       "      <th>24</th>\n",
       "      <th>25</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-1.020350</td>\n",
       "      <td>1.284279</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-0.127737</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>0.679690</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.382245</td>\n",
       "      <td>-0.774151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>0.690023</td>\n",
       "      <td>0.282777</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>1.511598</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>0.679690</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.459799</td>\n",
       "      <td>0.643615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-1.020350</td>\n",
       "      <td>0.282777</td>\n",
       "      <td>1.771041</td>\n",
       "      <td>0.418708</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>0.679690</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.245196</td>\n",
       "      <td>-0.065268</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-1.020350</td>\n",
       "      <td>1.284279</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-0.127737</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>1.408395</td>\n",
       "      <td>-1.277194</td>\n",
       "      <td>0.213475</td>\n",
       "      <td>-0.065268</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>2.400396</td>\n",
       "      <td>-1.720227</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>1.511598</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>-0.049015</td>\n",
       "      <td>-1.277194</td>\n",
       "      <td>-0.024889</td>\n",
       "      <td>2.061382</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3905</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-0.653841</td>\n",
       "      <td>1.284279</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-0.674182</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>-1.506426</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.536973</td>\n",
       "      <td>0.643615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3906</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.455047</td>\n",
       "      <td>-0.898180</td>\n",
       "      <td>-0.718725</td>\n",
       "      <td>1.771041</td>\n",
       "      <td>-1.220627</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>1.408395</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>1.529609</td>\n",
       "      <td>-0.065268</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3907</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.455047</td>\n",
       "      <td>1.545210</td>\n",
       "      <td>0.282777</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>2.058043</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>-0.777720</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.360576</td>\n",
       "      <td>0.643615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3908</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.455047</td>\n",
       "      <td>1.789549</td>\n",
       "      <td>1.284279</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-0.127737</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>-1.506426</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.252799</td>\n",
       "      <td>0.643615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3909</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-0.776011</td>\n",
       "      <td>0.282777</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-1.220627</td>\n",
       "      <td>1.581280</td>\n",
       "      <td>-0.049015</td>\n",
       "      <td>-1.277194</td>\n",
       "      <td>-1.082511</td>\n",
       "      <td>-1.483035</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3910 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       0    1    2    3    4    5    6    7    8    9   ...        16  \\\n",
       "0     1.0  0.0  0.0  1.0  1.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "1     1.0  0.0  1.0  0.0  1.0  0.0  0.0  0.0  0.0  1.0  ... -0.721400   \n",
       "2     1.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "3     1.0  0.0  1.0  0.0  1.0  1.0  0.0  0.0  0.0  1.0  ... -0.721400   \n",
       "4     0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "...   ...  ...  ...  ...  ...  ...  ...  ...  ...  ...  ...       ...   \n",
       "3905  1.0  0.0  0.0  1.0  1.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "3906  1.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  1.0  0.0  ...  1.455047   \n",
       "3907  0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  ...  1.455047   \n",
       "3908  1.0  0.0  0.0  1.0  0.0  1.0  0.0  0.0  0.0  1.0  ...  1.455047   \n",
       "3909  0.0  0.0  1.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "\n",
       "            17        18        19        20        21        22        23  \\\n",
       "0    -1.020350  1.284279 -0.725271 -0.127737 -0.632399  0.679690  0.782966   \n",
       "1     0.690023  0.282777 -0.725271  1.511598 -0.632399  0.679690  0.782966   \n",
       "2    -1.020350  0.282777  1.771041  0.418708 -0.632399  0.679690  0.782966   \n",
       "3    -1.020350  1.284279 -0.725271 -0.127737 -0.632399  1.408395 -1.277194   \n",
       "4     2.400396 -1.720227 -0.725271  1.511598 -0.632399 -0.049015 -1.277194   \n",
       "...        ...       ...       ...       ...       ...       ...       ...   \n",
       "3905 -0.653841  1.284279 -0.725271 -0.674182 -0.632399 -1.506426  0.782966   \n",
       "3906 -0.898180 -0.718725  1.771041 -1.220627 -0.632399  1.408395  0.782966   \n",
       "3907  1.545210  0.282777 -0.725271  2.058043 -0.632399 -0.777720  0.782966   \n",
       "3908  1.789549  1.284279 -0.725271 -0.127737 -0.632399 -1.506426  0.782966   \n",
       "3909 -0.776011  0.282777 -0.725271 -1.220627  1.581280 -0.049015 -1.277194   \n",
       "\n",
       "            24        25  \n",
       "0    -0.382245 -0.774151  \n",
       "1    -0.459799  0.643615  \n",
       "2    -0.245196 -0.065268  \n",
       "3     0.213475 -0.065268  \n",
       "4    -0.024889  2.061382  \n",
       "...        ...       ...  \n",
       "3905 -0.536973  0.643615  \n",
       "3906  1.529609 -0.065268  \n",
       "3907 -0.360576  0.643615  \n",
       "3908 -0.252799  0.643615  \n",
       "3909 -1.082511 -1.483035  \n",
       "\n",
       "[3910 rows x 26 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "## apply tansformation on test(transform)\n",
    "X_test=preprocessor.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.        ,  0.        ,  0.        , ..., -1.2771941 ,\n",
       "        -0.73751038, -0.77415132],\n",
       "       [ 1.        ,  0.        ,  0.        , ..., -1.2771941 ,\n",
       "        -0.6704111 , -0.06526803],\n",
       "       [ 1.        ,  0.        ,  0.        , ...,  0.78296635,\n",
       "        -0.4208322 , -0.77415132],\n",
       "       ...,\n",
       "       [ 0.        ,  1.        ,  0.        , ...,  0.78296635,\n",
       "         0.69001249,  0.64361526],\n",
       "       [ 1.        ,  0.        ,  0.        , ...,  0.78296635,\n",
       "        -0.22827818, -0.77415132],\n",
       "       [ 1.        ,  1.        ,  0.        , ...,  0.78296635,\n",
       "        -0.44611323,  2.06138184]])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adaboost Classifier Training\n",
    "#### We can also combine multiple algorithms\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "      <th>19</th>\n",
       "      <th>20</th>\n",
       "      <th>21</th>\n",
       "      <th>22</th>\n",
       "      <th>23</th>\n",
       "      <th>24</th>\n",
       "      <th>25</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-1.020350</td>\n",
       "      <td>1.284279</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-0.127737</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>0.679690</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.382245</td>\n",
       "      <td>-0.774151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>0.690023</td>\n",
       "      <td>0.282777</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>1.511598</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>0.679690</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.459799</td>\n",
       "      <td>0.643615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-1.020350</td>\n",
       "      <td>0.282777</td>\n",
       "      <td>1.771041</td>\n",
       "      <td>0.418708</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>0.679690</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.245196</td>\n",
       "      <td>-0.065268</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-1.020350</td>\n",
       "      <td>1.284279</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-0.127737</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>1.408395</td>\n",
       "      <td>-1.277194</td>\n",
       "      <td>0.213475</td>\n",
       "      <td>-0.065268</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>2.400396</td>\n",
       "      <td>-1.720227</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>1.511598</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>-0.049015</td>\n",
       "      <td>-1.277194</td>\n",
       "      <td>-0.024889</td>\n",
       "      <td>2.061382</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3905</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-0.653841</td>\n",
       "      <td>1.284279</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-0.674182</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>-1.506426</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.536973</td>\n",
       "      <td>0.643615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3906</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.455047</td>\n",
       "      <td>-0.898180</td>\n",
       "      <td>-0.718725</td>\n",
       "      <td>1.771041</td>\n",
       "      <td>-1.220627</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>1.408395</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>1.529609</td>\n",
       "      <td>-0.065268</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3907</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.455047</td>\n",
       "      <td>1.545210</td>\n",
       "      <td>0.282777</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>2.058043</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>-0.777720</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.360576</td>\n",
       "      <td>0.643615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3908</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.455047</td>\n",
       "      <td>1.789549</td>\n",
       "      <td>1.284279</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-0.127737</td>\n",
       "      <td>-0.632399</td>\n",
       "      <td>-1.506426</td>\n",
       "      <td>0.782966</td>\n",
       "      <td>-0.252799</td>\n",
       "      <td>0.643615</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3909</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.721400</td>\n",
       "      <td>-0.776011</td>\n",
       "      <td>0.282777</td>\n",
       "      <td>-0.725271</td>\n",
       "      <td>-1.220627</td>\n",
       "      <td>1.581280</td>\n",
       "      <td>-0.049015</td>\n",
       "      <td>-1.277194</td>\n",
       "      <td>-1.082511</td>\n",
       "      <td>-1.483035</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3910 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       0    1    2    3    4    5    6    7    8    9   ...        16  \\\n",
       "0     1.0  0.0  0.0  1.0  1.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "1     1.0  0.0  1.0  0.0  1.0  0.0  0.0  0.0  0.0  1.0  ... -0.721400   \n",
       "2     1.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "3     1.0  0.0  1.0  0.0  1.0  1.0  0.0  0.0  0.0  1.0  ... -0.721400   \n",
       "4     0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "...   ...  ...  ...  ...  ...  ...  ...  ...  ...  ...  ...       ...   \n",
       "3905  1.0  0.0  0.0  1.0  1.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "3906  1.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  1.0  0.0  ...  1.455047   \n",
       "3907  0.0  0.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  0.0  ...  1.455047   \n",
       "3908  1.0  0.0  0.0  1.0  0.0  1.0  0.0  0.0  0.0  1.0  ...  1.455047   \n",
       "3909  0.0  0.0  1.0  0.0  1.0  0.0  0.0  0.0  0.0  0.0  ... -0.721400   \n",
       "\n",
       "            17        18        19        20        21        22        23  \\\n",
       "0    -1.020350  1.284279 -0.725271 -0.127737 -0.632399  0.679690  0.782966   \n",
       "1     0.690023  0.282777 -0.725271  1.511598 -0.632399  0.679690  0.782966   \n",
       "2    -1.020350  0.282777  1.771041  0.418708 -0.632399  0.679690  0.782966   \n",
       "3    -1.020350  1.284279 -0.725271 -0.127737 -0.632399  1.408395 -1.277194   \n",
       "4     2.400396 -1.720227 -0.725271  1.511598 -0.632399 -0.049015 -1.277194   \n",
       "...        ...       ...       ...       ...       ...       ...       ...   \n",
       "3905 -0.653841  1.284279 -0.725271 -0.674182 -0.632399 -1.506426  0.782966   \n",
       "3906 -0.898180 -0.718725  1.771041 -1.220627 -0.632399  1.408395  0.782966   \n",
       "3907  1.545210  0.282777 -0.725271  2.058043 -0.632399 -0.777720  0.782966   \n",
       "3908  1.789549  1.284279 -0.725271 -0.127737 -0.632399 -1.506426  0.782966   \n",
       "3909 -0.776011  0.282777 -0.725271 -1.220627  1.581280 -0.049015 -1.277194   \n",
       "\n",
       "            24        25  \n",
       "0    -0.382245 -0.774151  \n",
       "1    -0.459799  0.643615  \n",
       "2    -0.245196 -0.065268  \n",
       "3     0.213475 -0.065268  \n",
       "4    -0.024889  2.061382  \n",
       "...        ...       ...  \n",
       "3905 -0.536973  0.643615  \n",
       "3906  1.529609 -0.065268  \n",
       "3907 -0.360576  0.643615  \n",
       "3908 -0.252799  0.643615  \n",
       "3909 -1.082511 -1.483035  \n",
       "\n",
       "[3910 rows x 26 columns]"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3995    0\n",
       "2610    0\n",
       "3083    0\n",
       "3973    0\n",
       "4044    0\n",
       "       ..\n",
       "4426    0\n",
       "466     0\n",
       "3092    0\n",
       "3772    0\n",
       "860     1\n",
       "Name: ProdTaken, Length: 3910, dtype: int64"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score, classification_report,ConfusionMatrixDisplay, \\\n",
    "                            precision_score, recall_score, f1_score, roc_auc_score,roc_curve "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logisitic Regression\n",
      "Model performance for Training set\n",
      "- Accuracy: 0.8458\n",
      "- F1 score: 0.8200\n",
      "- Precision: 0.6994\n",
      "- Recall: 0.3032\n",
      "- Roc Auc Score: 0.6366\n",
      "----------------------------------\n",
      "Model performance for Test set\n",
      "- Accuracy: 0.8354\n",
      "- F1 score: 0.8078\n",
      "- Precision: 0.6829\n",
      "- Recall: 0.2932\n",
      "- Roc Auc Score: 0.6301\n",
      "===================================\n",
      "\n",
      "\n",
      "Decision Tree\n",
      "Model performance for Training set\n",
      "- Accuracy: 1.0000\n",
      "- F1 score: 1.0000\n",
      "- Precision: 1.0000\n",
      "- Recall: 1.0000\n",
      "- Roc Auc Score: 1.0000\n",
      "----------------------------------\n",
      "Model performance for Test set\n",
      "- Accuracy: 0.9192\n",
      "- F1 score: 0.9183\n",
      "- Precision: 0.8111\n",
      "- Recall: 0.7644\n",
      "- Roc Auc Score: 0.8606\n",
      "===================================\n",
      "\n",
      "\n",
      "Random Forest\n",
      "Model performance for Training set\n",
      "- Accuracy: 1.0000\n",
      "- F1 score: 1.0000\n",
      "- Precision: 1.0000\n",
      "- Recall: 1.0000\n",
      "- Roc Auc Score: 1.0000\n",
      "----------------------------------\n",
      "Model performance for Test set\n",
      "- Accuracy: 0.9315\n",
      "- F1 score: 0.9265\n",
      "- Precision: 0.9697\n",
      "- Recall: 0.6702\n",
      "- Roc Auc Score: 0.8325\n",
      "===================================\n",
      "\n",
      "\n",
      "Gradient Boost\n",
      "Model performance for Training set\n",
      "- Accuracy: 0.8939\n",
      "- F1 score: 0.8819\n",
      "- Precision: 0.8756\n",
      "- Recall: 0.5021\n",
      "- Roc Auc Score: 0.7429\n",
      "----------------------------------\n",
      "Model performance for Test set\n",
      "- Accuracy: 0.8589\n",
      "- F1 score: 0.8398\n",
      "- Precision: 0.7732\n",
      "- Recall: 0.3927\n",
      "- Roc Auc Score: 0.6824\n",
      "===================================\n",
      "\n",
      "\n",
      "Adaboost\n",
      "Model performance for Training set\n",
      "- Accuracy: 0.8565\n",
      "- F1 score: 0.8365\n",
      "- Precision: 0.7308\n",
      "- Recall: 0.3649\n",
      "- Roc Auc Score: 0.6670\n",
      "----------------------------------\n",
      "Model performance for Test set\n",
      "- Accuracy: 0.8354\n",
      "- F1 score: 0.8115\n",
      "- Precision: 0.6630\n",
      "- Recall: 0.3194\n",
      "- Roc Auc Score: 0.6400\n",
      "===================================\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "models={\n",
    "    \"Logisitic Regression\":LogisticRegression(),\n",
    "    \"Decision Tree\":DecisionTreeClassifier(),\n",
    "    \"Random Forest\":RandomForestClassifier(),\n",
    "    \"Gradient Boost\":GradientBoostingClassifier(),\n",
    "    \"Adaboost\":AdaBoostClassifier()\n",
    "}\n",
    "for i in range(len(list(models))):\n",
    "    model = list(models.values())[i]\n",
    "    model.fit(X_train, y_train) # Train model\n",
    "\n",
    "    # Make predictions\n",
    "    y_train_pred = model.predict(X_train)\n",
    "    y_test_pred = model.predict(X_test)\n",
    "\n",
    "    # Training set performance\n",
    "    model_train_accuracy = accuracy_score(y_train, y_train_pred) # Calculate Accuracy\n",
    "    model_train_f1 = f1_score(y_train, y_train_pred, average='weighted') # Calculate F1-score\n",
    "    model_train_precision = precision_score(y_train, y_train_pred) # Calculate Precision\n",
    "    model_train_recall = recall_score(y_train, y_train_pred) # Calculate Recall\n",
    "    model_train_rocauc_score = roc_auc_score(y_train, y_train_pred)\n",
    "\n",
    "\n",
    "    # Test set performance\n",
    "    model_test_accuracy = accuracy_score(y_test, y_test_pred) # Calculate Accuracy\n",
    "    model_test_f1 = f1_score(y_test, y_test_pred, average='weighted') # Calculate F1-score\n",
    "    model_test_precision = precision_score(y_test, y_test_pred) # Calculate Precision\n",
    "    model_test_recall = recall_score(y_test, y_test_pred) # Calculate Recall\n",
    "    model_test_rocauc_score = roc_auc_score(y_test, y_test_pred) #Calculate Roc\n",
    "\n",
    "\n",
    "    print(list(models.keys())[i])\n",
    "    \n",
    "    print('Model performance for Training set')\n",
    "    print(\"- Accuracy: {:.4f}\".format(model_train_accuracy))\n",
    "    print('- F1 score: {:.4f}'.format(model_train_f1))\n",
    "    \n",
    "    print('- Precision: {:.4f}'.format(model_train_precision))\n",
    "    print('- Recall: {:.4f}'.format(model_train_recall))\n",
    "    print('- Roc Auc Score: {:.4f}'.format(model_train_rocauc_score))\n",
    "\n",
    "    \n",
    "    \n",
    "    print('----------------------------------')\n",
    "    \n",
    "    print('Model performance for Test set')\n",
    "    print('- Accuracy: {:.4f}'.format(model_test_accuracy))\n",
    "    print('- F1 score: {:.4f}'.format(model_test_f1))\n",
    "    print('- Precision: {:.4f}'.format(model_test_precision))\n",
    "    print('- Recall: {:.4f}'.format(model_test_recall))\n",
    "    print('- Roc Auc Score: {:.4f}'.format(model_test_rocauc_score))\n",
    "\n",
    "    \n",
    "    print('='*35)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Hyperparameter Training\n",
    "rf_params = {\"max_depth\": [5, 8, 15, None, 10],\n",
    "             \"max_features\": [5, 7, \"auto\", 8],\n",
    "             \"min_samples_split\": [2, 8, 15, 20],\n",
    "             \"n_estimators\": [100, 200, 500, 1000]}\n",
    "adaboost_param={\n",
    "    \"n_estimators\":[50,60,70,80,90],\n",
    "    \"algorithm\":['SAMME','SAMME.R']\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'max_depth': [5, 8, 15, None, 10],\n",
       " 'max_features': [5, 7, 'auto', 8],\n",
       " 'min_samples_split': [2, 8, 15, 20],\n",
       " 'n_estimators': [100, 200, 500, 1000]}"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'n_estimators': [50, 60, 70, 80, 90], 'algorithm': ['SAMME', 'SAMME.R']}"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adaboost_param"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Models list for Hyperparameter tuning\n",
    "randomcv_models = [\n",
    "                   (\"RF\", RandomForestClassifier(), rf_params),\n",
    "    (\"AB\", AdaBoostClassifier(), adaboost_param)\n",
    "                   \n",
    "                   ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('RF',\n",
       "  RandomForestClassifier(),\n",
       "  {'max_depth': [5, 8, 15, None, 10],\n",
       "   'max_features': [5, 7, 'auto', 8],\n",
       "   'min_samples_split': [2, 8, 15, 20],\n",
       "   'n_estimators': [100, 200, 500, 1000]}),\n",
       " ('AB',\n",
       "  AdaBoostClassifier(),\n",
       "  {'n_estimators': [50, 60, 70, 80, 90], 'algorithm': ['SAMME', 'SAMME.R']})]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "randomcv_models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 3 folds for each of 100 candidates, totalling 300 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  98 tasks      | elapsed:    5.3s\n",
      "[Parallel(n_jobs=-1)]: Done 300 out of 300 | elapsed:   18.1s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 3 folds for each of 10 candidates, totalling 30 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=-1)]: Using backend LokyBackend with 32 concurrent workers.\n",
      "[Parallel(n_jobs=-1)]: Done  15 out of  30 | elapsed:    0.2s remaining:    0.2s\n",
      "[Parallel(n_jobs=-1)]: Done  30 out of  30 | elapsed:    0.3s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------- Best Params for RF -------------------\n",
      "{'n_estimators': 100, 'min_samples_split': 2, 'max_features': 8, 'max_depth': 15}\n",
      "---------------- Best Params for AB -------------------\n",
      "{'n_estimators': 80, 'algorithm': 'SAMME'}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import RandomizedSearchCV\n",
    "\n",
    "model_param = {}\n",
    "for name, model, params in randomcv_models:\n",
    "    random = RandomizedSearchCV(estimator=model,\n",
    "                                   param_distributions=params,\n",
    "                                   n_iter=100,\n",
    "                                   cv=3,\n",
    "                                   verbose=2,\n",
    "                                   n_jobs=-1)\n",
    "    random.fit(X_train, y_train)\n",
    "    model_param[name] = random.best_params_\n",
    "\n",
    "for model_name in model_param:\n",
    "    print(f\"---------------- Best Params for {model_name} -------------------\")\n",
    "    print(model_param[model_name])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random Forest\n",
      "Model performance for Training set\n",
      "- Accuracy: 1.0000\n",
      "- F1 score: 1.0000\n",
      "- Precision: 1.0000\n",
      "- Recall: 1.0000\n",
      "- Roc Auc Score: 1.0000\n",
      "----------------------------------\n",
      "Model performance for Test set\n",
      "- Accuracy: 0.9356\n",
      "- F1 score: 0.9313\n",
      "- Precision: 0.9706\n",
      "- Recall: 0.6911\n",
      "- Roc Auc Score: 0.8430\n",
      "===================================\n",
      "\n",
      "\n",
      "Adaboost\n",
      "Model performance for Training set\n",
      "- Accuracy: 0.8465\n",
      "- F1 score: 0.8132\n",
      "- Precision: 0.7699\n",
      "- Recall: 0.2524\n",
      "- Roc Auc Score: 0.6176\n",
      "----------------------------------\n",
      "Model performance for Test set\n",
      "- Accuracy: 0.8364\n",
      "- F1 score: 0.7977\n",
      "- Precision: 0.7818\n",
      "- Recall: 0.2251\n",
      "- Roc Auc Score: 0.6049\n",
      "===================================\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "models={\n",
    "    \n",
    "    \"Random Forest\":RandomForestClassifier(n_estimators=1000,min_samples_split=2,\n",
    "                                          max_features=7,max_depth=None),\n",
    "    \"Adaboost\":AdaBoostClassifier(n_estimators=80, algorithm='SAMME')\n",
    "}\n",
    "for i in range(len(list(models))):\n",
    "    model = list(models.values())[i]\n",
    "    model.fit(X_train, y_train) # Train model\n",
    "\n",
    "    # Make predictions\n",
    "    y_train_pred = model.predict(X_train)\n",
    "    y_test_pred = model.predict(X_test)\n",
    "\n",
    "    # Training set performance\n",
    "    model_train_accuracy = accuracy_score(y_train, y_train_pred) # Calculate Accuracy\n",
    "    model_train_f1 = f1_score(y_train, y_train_pred, average='weighted') # Calculate F1-score\n",
    "    model_train_precision = precision_score(y_train, y_train_pred) # Calculate Precision\n",
    "    model_train_recall = recall_score(y_train, y_train_pred) # Calculate Recall\n",
    "    model_train_rocauc_score = roc_auc_score(y_train, y_train_pred)\n",
    "\n",
    "\n",
    "    # Test set performance\n",
    "    model_test_accuracy = accuracy_score(y_test, y_test_pred) # Calculate Accuracy\n",
    "    model_test_f1 = f1_score(y_test, y_test_pred, average='weighted') # Calculate F1-score\n",
    "    model_test_precision = precision_score(y_test, y_test_pred) # Calculate Precision\n",
    "    model_test_recall = recall_score(y_test, y_test_pred) # Calculate Recall\n",
    "    model_test_rocauc_score = roc_auc_score(y_test, y_test_pred) #Calculate Roc\n",
    "\n",
    "\n",
    "    print(list(models.keys())[i])\n",
    "    \n",
    "    print('Model performance for Training set')\n",
    "    print(\"- Accuracy: {:.4f}\".format(model_train_accuracy))\n",
    "    print('- F1 score: {:.4f}'.format(model_train_f1))\n",
    "    \n",
    "    print('- Precision: {:.4f}'.format(model_train_precision))\n",
    "    print('- Recall: {:.4f}'.format(model_train_recall))\n",
    "    print('- Roc Auc Score: {:.4f}'.format(model_train_rocauc_score))\n",
    "\n",
    "    \n",
    "    \n",
    "    print('----------------------------------')\n",
    "    \n",
    "    print('Model performance for Test set')\n",
    "    print('- Accuracy: {:.4f}'.format(model_test_accuracy))\n",
    "    print('- F1 score: {:.4f}'.format(model_test_f1))\n",
    "    print('- Precision: {:.4f}'.format(model_test_precision))\n",
    "    print('- Recall: {:.4f}'.format(model_test_recall))\n",
    "    print('- Roc Auc Score: {:.4f}'.format(model_test_rocauc_score))\n",
    "\n",
    "    \n",
    "    print('='*35)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABLR0lEQVR4nO3dd3gUZdfA4d8h9F6CSJXeOwgqgmDHgogCKhZsvCCoYG+fIjZULCAqor6iotgLIr6KSpMiNfTeQ9EQOgRIOd8fzyQsIdlsQjabTc59XXtld+qZSTJnZ54mqooxxhiTngKhDsAYY0zuZonCGGOMX5YojDHG+GWJwhhjjF+WKIwxxvhlicIYY4xflihMpojIChHpHOo4cgsReUJEPgjRvseJyPOh2Hd2E5E+IvJbFte1v8kgs0QRxkRks4jEicghEdnlXThKBnOfqtpEVacFcx/JRKSIiLwkIlu941wnIg+LiOTE/tOIp7OIRPtOU9UXVfWuIO1PROQ+EVkuIodFJFpEvhaRZsHYX1aJyFARGX8621DVz1T10gD2dUpyzMm/yfzKEkX4u1pVSwItgVbA46ENJ/NEpGA6s74GLgKuAEoBtwD9gJFBiEFEJLf9P4wE7gfuA8oD9YEfgCuze0d+fgdBF8p9mwCpqr3C9AVsBi72+fwK8LPP53OA2cA+YAnQ2WdeeeAjYAewF/jBZ95VQJS33mygeep9AlWAOKC8z7xWwG6gkPf5DmCVt/1fgbN8llVgILAO2JTGsV0EHAWqp5reHkgE6nqfpwEvAfOA/cCPqWLydw6mAS8As7xjqQvc7sV8ENgI/MdbtoS3TBJwyHtVAYYC471lanrHdRuw1TsXT/rsrxjwsXc+VgGPANHp/G7recfZzs/vfxzwNvCzF+/fQB2f+SOBbcABYCHQ0WfeUOAbYLw3/y6gHTDHO1c7gdFAYZ91mgBTgD3AP8ATwOXAcSDeOydLvGXLAB9629kOPA9EePP6euf8DW9bz3vT/vLmizfvX+93uhRoivuSEO/t7xDwU+r/AyDCi2uDd04WkupvyF5ZuNaEOgB7ncYv7+R/kGrAMmCk97kqEIv7Nl4AuMT7XNGb/zPwJVAOKARc4E1v7f2Dtvf+6W7z9lMkjX3+CdztE8+rwBjvfXdgPdAIKAg8Bcz2WVa9i055oFgaxzYcmJ7OcW/hxAV8mnchaoq7mH/LiQt3RudgGu6C3sSLsRDu23od72J1AXAEaO0t35lUF3bSThTv45JCC+AY0Mj3mLxzXg13AUwvUfQHtmTw+x+Hu9C28+L/DPjCZ/7NQAVv3oPALqCoT9zx3u+pgBdvG1xiLegdyypgsLd8KdxF/0GgqPe5fepz4LPvH4D3vN/JGbhEnvw76wskAPd6+yrGyYniMtwFvqz3e2gEVPY55uf9/B88jPs/aOCt2wKoEOr/1XB/hTwAe53GL8/9gxzCfXNS4A+grDfvUeDTVMv/irvwV8Z9My6XxjbfBZ5LNW0NJxKJ7z/lXcCf3nvBfXvt5H3+BbjTZxsFcBfds7zPClzo59g+8L3opZo3F++bOu5iP9xnXmPcN84If+fAZ91hGZzjH4D7vfedCSxRVPOZPw+4wXu/EbjMZ95dqbfnM+9JYG4GsY0DPvD5fAWw2s/ye4EWPnHPyGD7g4Hvvfc3AovTWS7lHHifK+ESZDGfaTcCU733fYGtqbbRlxOJ4kJgLS5pFUjjmP0lijXANaf7v2Wvk1+57ZmsybzuqloKdxFrCER6088CeorIvuQXcD4uSVQH9qjq3jS2dxbwYKr1quMes6T2DXCuiFQBOuEukjN9tjPSZxt7cMmkqs/62/wc124v1rRU9uantZ0tuDuDSPyfgzRjEJGuIjJXRPZ4y1/BiXMaqF0+748AyRUMqqTan7/jjyX94w9kX4jIgyKySkT2e8dShpOPJfWx1xeRSV7FiAPAiz7LV8c9zgnEWbjfwU6f8/4e7s4izX37UtU/cY+93gb+EZGxIlI6wH1nJk4TIEsUeYSqTsd92xrhTdqG+zZd1udVQlWHe/PKi0jZNDa1DXgh1XrFVXVCGvvcB/wG9AJuAiao97XO285/Um2nmKrO9t2En0P6HWgvItV9J4pIO9zF4E+fyb7L1MA9UtmdwTk4JQYRKYJ7dDUCqKSqZYHJuASXUbyB2Il75JRW3Kn9AVQTkbZZ2ZGIdMTdUfXC3TmWxT3v960xlvp43gVWA/VUtTTuWX/y8ttwj+TSkno723B3FJE+5720qjbxs87JG1QdpaptcI8F6+MeKWW4XgZxmiyyRJG3vAlcIiItcYWUV4vIZSISISJFveqd1VR1J+7R0DsiUk5EColIJ28b7wP9RaS9VxOohIhcKSKl0tnn58CtwHXe+2RjgMdFpAmAiJQRkZ6BHoiq/o67WH4rIk28YzgH9xz+XVVd57P4zSLSWESKA8OAb1Q10d85SGe3hYEiQAyQICJdAd8qm/8AFUSkTKDHkcpXuHNSTkSqAoPSW9A7vneACV7Mhb34bxCRxwLYVylcOUAMUFBEngYy+lZeClewfUhEGgIDfOZNAs4UkcFeteVSItLem/cPUDO51pj39/Ub8JqIlBaRAiJSR0QuCCBuRORs7++vEHAYV6kh0Wdftf2s/gHwnIjU8/5+m4tIhUD2a9JniSIPUdUY4BPg/1R1G3AN7lthDO6b1sOc+J3fgvvmvRpXeD3Y28YC4G7crf9eXIF0Xz+7nYirofOPqi7xieV74GXgC+8xxnKgayYP6TpgKvA/XFnMeFxNmntTLfcp7m5qF66g9T4vhozOwUlU9aC37le4Y7/JO77k+auBCcBG75FKWo/j/BkGRAObcHdM3+C+eafnPk48gtmHe6RyLfBTAPv6FfdlYC3ucdxR/D/qAngId8wHcV8Yvkye4Z2bS4Crced5HdDFm/219zNWRBZ572/FJd6VuHP5DYE9SgOX0N731tuCewyXfKf8IdDYO/8/pLHu67jf32+4pPchrrDcnAY58aTAmPAjItNwBakhaR19OkRkAK6gO6Bv2saEit1RGJNDRKSyiHTwHsU0wFU1/T7UcRmTEWsRaUzOKYyr/VML9yjpC1w5hDG5mj16MsYY45c9ejLGGONX2D16ioyM1Jo1a4Y6DGOMCSsLFy7craoVs7Ju2CWKmjVrsmDBglCHYYwxYUVEtmR1XXv0ZIwxxi9LFMYYY/yyRGGMMcYvSxTGGGP8skRhjDHGL0sUxhhj/ApaohCR/4rIvyKyPJ35IiKjRGS9iCwVkdbBisUYY0zWBbMdxThcF8mfpDO/K6576nq48Znf9X4aY0y+oKocS0gKdRgZClqiUNUZIlLTzyLXAJ94I6LNFZGyIlLZG/TEGGPyvGGTVvLRrM1B3Uf7rcu4c8GPp7WNULbMrsrJA6lEe9NOSRQi0g/oB1CjRo0cCc4YYwK1etcB+rz/N3HxiRkv7ONofCKVyxTl1nNrZntMRffFcu6Yl2n463ccODO9QR0DE8pEIWlMS7MrW1UdC4wFaNu2rXV3a4wJqUlLd/DK/9ag3iUr7ngisYeP06N1VSqUKJypbZ1XN5IuDc7I/iCvewT+mAiPP07pp56CEiWyvKlQJopoTh5cvhqwI0SxGGNMwBZs3svO/XFc3fzEaLilixXiiSsaUbhgCCuTrlgBZctC1arw8sswbBg0aXLamw1lopgIDBKRL3CF2PutfMIYkxscOpbAk98v49DRhDTnr/nnIMUKRfB675Y5G1h6Dh+G556D116DPn1g3DioWzfbNh+0RCEiE4DOQKSIRAPPAIUAVHUMMBm4AlgPHAFuD1Ysxpi8ZWPMId6fuYnEpODUGNp96Dh/rv6XmhWKU7LoqZfJssUL0blBlnrszn4//wwDB8KWLXDHHe5OIpsFs9bTjRnMV2BgsPZvjAlPMQeP8emczRxPTL84cvHWvfy9aQ9nli6KpFXamQ3qVCzBuNvbUb188eDsIDu8845LEo0bw4wZ0LFjUHYTduNRGGNyv6PxiXyzMJqjmawFBLB46z5+XraTwhEF0q7y4qlfqST/u78TBQoEKVPkVgkJEBMDlStDr14QFwf33guFM1eInhmWKIwxWbLvyHGmrYkhSU/95r/mn4O8N31jlrddplgh5jx+IcUL2yXqJPPmwX/+AwULwty5EBkJDz4Y9N3ab8EYk2m79h/llV9X892i7ekuIwK/3N+RqmWLZXr7RQpGhLb2UG6zbx888QSMGePuJEaOhAI5d34sURhjMmXHvjj6fbqA5dsPUL5EYb6/57w0lytRpCCRJYvkcHR50LJlcMkl7nHTffe5Kq+lS+doCJYojMkH9h05nulWw2mJO57IZW/OID5R6Vgvkleub07lMpm/YzABiI+HQoWgfn3o0gUefhhah6bvVEsUxoS54xl0Krcl9jCXvTmDpGzs06DveTUZ2KUuFUvZHUO2O3bMVXEdPx4WLYKSJWHChJCGZInCmDD2wcyNPP/zqoCW7depNrUjs96NQ7KCEQW4vOmZlCxil49s9+efMGAArF0LvXu7pFGyZKijskRhTDi4YewcVmw/cMr0owmJlCgcwT1d/LfCLV44gpva16BIwYhghWhOR1wc9Ovn7iJq14b//Q8uuyzUUaWwRGFMLrcsej9zN+6hadXStKtZ4ZT5zauVoXurqiGIzGSbokVh92546ilXu6lY7ir3sURhTC72w+LtvD5lLQCDL6rPxY0rhTgik22WLnUF1B9+CNWqua44crDKa2bkzqiMMQD8tnIXMQeP0aNVVbo0DEJX1CbnHT58ogbTokWwbp2bnkuTBFiiMCbXUlUmL9tF1XLFeL13SyLyW1cVedHEia5fphEjXAd+a9a4qq+5nD16MiaX2n3oOABJ2Vmv1YTWDz+4xnJ//QUdOoQ6moBZojAmyHbuj+OXZbvSHr7Rj+SxEO44v1b2B2VyRnw8jBrl7hpat3ZdbxQt6hrShRFLFMYE2bjZm7PcQZ4IVClbNJsjMjli7lzXgd/SpfDooy5RlCoV6qiyxBKFMUEQdzyRxVv3kqSwZfcRShSOYPbjF2V6OwULCCWsYVt42bsXHn8cxo51Q5J+/z1cc02oozot9hdoTDZTVR77bik/Rp0YAv7M0kUpUyy8HjeYLBo7Fj74AIYMgaFDw/YuwpclCmOy2a8rdvFj1A4KFhAm9DsHIEtdbZswsmaN6931/PNh8GDo2hWaNw91VNnGEoUx2SAh8UTHfHsOxwPw3i1tOLtm+VCFZHLC0aPw0kswfDg0bAhRUVCkSJ5KEmCJwpjTNuqPdSmtp301rVomBNGYHDNlCtxzD6xfDzfdBK+9RtAG8A4xv4lCRIoCVwEdgSpAHLAc+FlVVwQ/PGNylzW7DnLrf//maPyJO4gjxxMoV7wQd3Q4UY21YqkinGFdcOddM2bApZdCvXouYVx8cagjCqp0E4WIDAWuBqYBfwP/AkWB+sBwL4k8qKpLgx+mMbnDpt2H+OfAMbq1qEL5EicGs29bsxxXNa8SwshM0CUmwsqV0KwZdOzo+mi66SbXLiKP83dHMV9Vh6Yz73UROQOokf0hGZP7Dehch0aVc3Y4ShNCixdD//6wapXrm6lSJdcFRz6Rbl9Pqvqz72cRKZFq/r+quiBYgRljTMgdPAgPPABt28LmzfDuu3BG/uucMcPCbBE5D/gAKAnUEJEWwH9U9Z5gB2dMqC3fvp8J87amdL+xbc+RkMZjctD+/e4x07ZtroX1Sy9BuXKhjiokAqn19AZwGTARQFWXiEinoEZlTAgs376f31bsOmna9LUxLIneT2TJEwXTdSqWoHKZvP9cOt86cMB13FemjBt17qKL4NxzQx1VSAVUPVZVt8nJ1b4SgxOOMaHz7rQN/Lxs5yk1HNvVKs9X/8nfF4p8IT4e3ngDnn8epk1zfTM99VSoo8oVAkkU27zHTyoihYH7gMBGczcmF9m25whR2/alOz967xEaVCrFr0PshjnfmTXLFVYvXw7du0PFiqGOKFcJJFH0B0YCVYFo4DfAyidMWNhz+Dirdx4A4OX/rWZJ9H6/y7evZS2p851774XRo6F6dfjxR+jWLdQR5TqBJIoGqtrHd4KIdABmBSckY07fgaPx7Nx3lGGTVjBrfWzK9PPrRjK0W+N016tifTLlD6onWlGfeSY89BA88wyULBnauHKpQBLFW0DrAKYZky32x8VzPCEp4wX9uO2/81jp3Um0qF6WJ7o2BKDBmaUoW7ywv1VNXrd6tXvMNGSI6/77ySdDHVGu569l9rnAeUBFEXnAZ1ZpICLYgZn8R1WJ2raPHu/ORrNh9M/z60ZyU/saNKtahurli5/+Bk14i4uDF1+El1+GEiXcZxMQf3cUhXFtJwoCvh2qHwCuD2TjInI5rnwjAvhAVYenml8GGI9r4V0QGKGqHwUcvckzVJWLXpvOxt2HAbj3wrqcUfr0qqB2rl/REoRx/vjDtYXYsAFuuQVGjMiXDeeyKt1EoarTgekiMk5Vt2R2wyISAbwNXIIrBJ8vIhNVdaXPYgOBlap6tYhUBNaIyGeqejyz+zPh7cGvlrBx92HOq1OBixtV4rbzahJRIG/2xGlCIDoaChZ0CePCC0MdTdgJpIziiIi8CjTBdQoIgKpmdLbbAetVdSOAiHwBXAP4JgoFSolrpFES2AMkBB6+ySvmb9kDwIvXNqNmZIkMljYmA4mJMGYMFC4Md98Nt94KN9zgxoowmZZuX08+PgNWA7WAZ4HNwPwA1qsKbPP5HO1N8zUaaATsAJYB96vqKaWYItJPRBaIyIKYmJgAdm3CTQERuresYknCnL5Fi+Ccc2DQIPj1VzdNxJLEaQgkUVRQ1Q+BeFWdrqp3AOcEsF5azw1SF1FeBkThxrpoCYwWkVO65FTVsaraVlXbVrSGMMaYtBw4APffD2ef7fpnmjABvv461FHlCYEkinjv504RuVJEWgHVAlgvGqju87ka7s7B1+3Ad+qsBzYBDQPYtslDDh6NZ0usdbZnTtOSJa7hXP/+rgrsDTfk2RHnclogZRTPe7WTHsS1nygNDA5gvflAPRGpBWwHbgBuSrXMVuAiYKaIVAIaABsDC92Es6PxiXwyZzOHjyWyJdbVdCpVtFCIozJhZ9MmmDrVjQ3RsaMblrRWrYzXM5mSYaJQ1Une2/1AF0hpmZ3RegkiMgj4FVc99r+qukJE+nvzxwDPAeNEZBnuUdWjqro7S0diwsa/B4/y/oyNvD9zU8q0UkUKMqBznRBGZcLK8eNujOphw9wIc9de67oAtyQRFP4a3EUAvXAF0P9T1eUichXwBFAMaJXRxlV1MjA51bQxPu93AJdmLXQTrj6evZn3Z26iYAHhtyGdqF3Ruk0wmTBzpnu8tHIl9OgBI0fm23Eicoq/O4oPcWUM84BRIrIFOBd4TFV/yIHYTB6jqqzYcYAtsUcoUrAAC5662B43mcyJiYFLL3VDkf70E1x1Vagjyhf8JYq2QHNVTRKRosBuoK6q7vKzjjHpWrHjAFe99RcAkSULW5IwgVGF33+HSy5x3X9PmuSqv5awqtQ5xV+tp+PJbRpU9Siw1pKEOR3/HjwKwJNXNOK7ARkWcxkDK1bABRe4u4hp09y0iy6yJJHD/N1RNBSRpd57Aep4nwVQVW0e9OhMnjF+7hae+mE54HpzrVHB+mAyfhw54kaae/VVNyzpBx9AJxtQKlT8JYpGORaFybNG/7mO16asRRUKFhBe6tGMVjXKhjosk5upQpcuMG8e3HabSxbW0Dak/HUKmOmOAI1Jbd2/hyhdtBC3nXsWdc4oyTUtU/fiYoxn507Xo2tEBDzxBJQpA507hzoqQ2Ats405LeWKF+KBSxtYkjBpS0yEUaOgQQN45x037ZprLEnkIoG0zDYmQ3HHE3ngqyj2HYk/afq6fw9Ssoj9mZl0LFjgxolYtAguuwyuuCLUEZk0BHRHISLFRKRBsIMx4WvLnsP8snwX/x48SmKSprxqR5bkutaBdA1m8p1XXoF27dwjpy+/hF9+gTrWOj83yvCrnohcDYzAjXhXS0RaAsNUtVuQYzMhpKqMmb6Rfw4cDWj5PYfdWFMPXdqArs0qBzM0E85UISEBChVySWLgQFe7qUyZUEdm/AjkmcBQ3CBE0wBUNUpEagYvJBMKC7fsYdb62JTPcfGJvDttA0ULFaBIwcCGSK9Uugi1Klr9dpOODRvgnnugaVPXT1PnzlYOESYCSRQJqrpfrLvePOdYQiKTluzkaEIiH87clDJedbKCBYQxN7ehcwMbW9ichmPHXBXXF15wdxLXXBPqiEwmBZIolovITUCEiNQD7gNmBzcskxNmrd/Ng18vSfl8bauqjOjZ4qRlbNxqc1oWLoSbb3bjQ/TsCW++CVWqhDoqk0mBJIp7gSeBY8DnuG7Dnw9mUCZ4kjvmi4tPZOWOAwCMv7M99SuVpELJIpYYTPYqWdINHjR5MnTtGupoTBYFkigaqOqTuGRhwtzfm/Zww9i5J02rXr4YZ5QuGqKITJ6SlAQffQRz5rhuNxo0gOXLoYA12QpngSSK10WkMvA18IWqrghyTCaIDh9LAGDYNU2oHVmSMsUKcVYFK4A22WD5cjdOxKxZrl+mw4dd532WJMJehr9BVe0CdAZigLEiskxEngp2YCa4WlYvy/n1ImlWzaolmtN0+DA8+ii0auXKIj76yPX0aj285hkBpXpV3aWqo4D+QBTwdDCDMsaEkaNHXXK49VZYswb69nXlEibPCKTBXSOgN3A9EAt8ATwY5LhMNpu3aQ9DvozikPfoSbB/ZHMaoqNd/0wvvQQVKrg7ifLlQx2VCZJAyig+AiYAl3pjXJsws2v/UXq9NweAHq2qElmqCPXPtHGqTRYkJMBbb8HTT7vO/Hr3hjZtLEnkcRkmClU9JycCMcExZ0MsL05eBbgBg17r1QJrPGmy5O+/XQd+S5a4zvtGj4ZatUIdlckB6SYKEflKVXuJyDJAfWdhI9yFhW8WRvPOtPVs2n2YC+pXZOQNLS1JmKxJSoLbb4f9++Gbb6BHDyuHyEf83VHc7/28KicCMdkr7ngiD329hOKFI7i6eRVG3dgq1CGZcKPqksLll0OpUvDdd1C1qntv8hV/I9zt9N7eo6qP+s4TkZeBR09dywTb9LUxLN++P8PlovceAeDSxpV48wZLEiaT1q1zPbtOmQIjRsCDD0LDhqGOyoRIIIXZl3BqUuiaxjQTJEuj97HC627j1V/XpHTpnZGihQpw/8X1gxmayWuOHYOXX4YXX4QiRVw5RP/+oY7KhJi/MooBwD1AbRFZ6jOrFDAr2IGZE4Z8GcWGmBM9uw7oXIchASSAAgIFI6xVrMmEgQPhww/hhhvg9dehso0tYvzfUXwO/AK8BDzmM/2gqu4JalTmJPGJyuVNzmRotyaIwBmlilihtMk+//7rCqvPPNO1sO7Z0w1LaozH39dNVdXNwEDgoM8LEbFK0zmsWOEIzixTlEqli1qSMNkjKQnGjnUd993v1V2pV8+ShDlFRncUVwELcdVjfa9OCtQOYlzGc+hYAkmqGS9oTGYsXerKHubMcaPMPftsqCMyuZi/Wk9XeT+tRU2IfDBzI8//7BrLnVvb7iJMNvnmG1cGUa4cfPKJG1jI7lKNHxmWdIpIBxEp4b2/WUReF5EawQ8tf/tr3W6e/3kVBQSeurIRA7vUDXVIJtwdcDXn6NzZFVqvWQO33GJJwmQokCox7wJHRKQF8AiwBfg0qFHlc2NnbOCezxYCMLRbE+7qWJuakdZls8mirVvdONUXXeT6Z4qMhJEjrX8mE7BAEkWCqipwDTBSVUfiqshmSEQuF5E1IrJeRB5LZ5nOIhIlIitEZHrgoedN09b8y4uTV1OggHDfRfW45ZyzQh2SCVfx8a6xXKNG8Pvv0KuXa21tTCYF0uDuoIg8DtwCdBSRCKBQRit5y72Na7AXDcwXkYmqutJnmbLAO8DlqrpVRM7IwjHkKbM3xALwyGUNuam9PeEzWbRlC3Tr5gqtr77a9fh6ln3pMFkTyB1Fb+AYcIeq7gKqAq8GsF47YL2qblTV47hxLK5JtcxNwHequhVAVf8NOPI86LtF0fy5+l+KFYqwJGGyJvmO4cwzoVIl+P57+PFHSxLmtATSzfguEfkMOFtErgLmqeonAWy7KrDN53M00D7VMvWBQiIyDfc4a2SA2w5rM9bGMGNtzCnTv1u8nbjjiVzYMN/fWJnMUoXPPoM333TDkJYsCb/9FuqoTB4RyAh3vXB3ENNwbSneEpGHVfWbjFZNY1rqB6QFgTbARUAxYI6IzFXVtali6Af0A6hRI/y/aY/8Yx2Lt+6lWKGIU+bd07kO915ULwRRmbC1Zg0MGABTp0L79hAb6xKFMdkkkDKKJ4Gzkx8LiUhF4Hcgo0QRDVT3+VwNSD1CXjSwW1UPA4dFZAbQAjgpUajqWGAsQNu2bcO2NC5q2z7W7jpIzMFjdKgbyad3pr7BMiYTEhLguedg+HAoVgzefRf69YMC1r+XyV6BJIoCqcoOYgmsbGM+UE9EagHbgRtwZRK+fgRGi0hBoDDu0dQbAWw77CQkJtH97RN9KbY9q1wIozF5QkQEzJwJ11/vOvCrVCnUEZk8KpBE8T8R+RU3bja4wu3JGa2kqgkiMgj4FYgA/quqK0Skvzd/jKquEpH/AUuBJOADVV2elQPJ7Wau2w3Alc0q88SVjahUqkiIIzJhadcueOIJ1+VG9eoweTIULRrqqEweF0hh9sMi0gM4H1fuMFZVvw9k46o6mVRJRVXHpPr8KoHVogprcfGJgOsivGrZYiGOxoSdxETXgd/jj0NcHHTt6hKFJQmTA/yNR1EPGAHUAZYBD6nq9pwKLK8qZONDmMxavNh14Ddvnmtd/c47UN8GpDI5x99V67/AJOA6XA+yb+VIRMaYk40eDZs3u+qvU6ZYkjA5zt+jp1Kq+r73fo2ILMqJgIzJ91Thhx+gZk1o1cp1wzFihOvt1ZgQ8HdHUVREWolIaxFpDRRL9dlkwud/bw11CCYcbN7sut7o0cM1ngOXICxJmBDyd0exE3jd5/Mun88KXBisoPKaQ8cS+Gu9q/VUu6L1AmvSEB/vqrg++6xrBzFixIlR54wJMX8DF3XJyUDyKlWl3ycLAOjWoooVZpu0vfcePPYYdO/uugDPAz0QmLwjkHYU5jR8MmdLSo+wD1/WIMTRmFwlNtY9amrTBu6+G+rWhcsvD3VUxpzCEkU2iDueyNgZGzlyPOGUeZ/M2QLA+DvbU7188ZwOzeRGqm4I0oceglKlYO1aKFLEkoTJtSxRnIb9cfF8uzCa9TGH+PzvrRSOKJBmNztPX9WY8+tF5nyAJvdZtcp14Dd9Opx7LowZAwXt39DkboH0HitAH6C2qg7zxss+U1XnBT26XO63FbsYNsmNw1SkYAF+HdzJhiw16VuyBM4+2/XsOnYs3HmndeBnwkIgX2XewfXDdCEwDDgIfAucHcS4wsK2PUcA+PPBC6hWrjiFC9o/vUlDdDRUqwbNm7taTXfeCWfYmCMmfARyZWuvqgOBowCquhfX02u+99bU9QBUKFHEkoQ51Y4d0Lu3G7N6+3YQcX01WZIwYSaQq1u8N/61Qsp4FElBjSoMzNu0B1WoHVmCMsUzHELc5CeJia7bjUaN3DCkjzwCkVZGZcJXII+eRgHfA2eIyAvA9cBTQY0qlzt8LIFe780B4NGuDUMcjclVjh6FTp1g/ny45BLXgV/duqGOypjTEkg345+JyELccKUCdFfVVUGPLBc7ctx1GX5u7Qpc3MgGizG4ltWFCrluv7t0gQcecI+dJK0RgY0JLxk+evJqOR0BfgIm4oYstWajwBXNKxNRwC4E+ZoqfPONu2tY5PWb+fLLcMMNliRMnhHIo6efceUTAhQFagFrgCZBjMuY3G/jRhg0CH75xfXyalVdTR4VyKOnZr6fvZ5j/xO0iHK596ZvYOKSHaEOw4Ta66/Dk0+6xnJvvgkDB1rDOZNnZfovW1UXiUi+bENx6FgCL/2ymvIlCnNxozM4r06FUIdkQuXQIbjiCteBX7VqoY7GmKAKpGX2Az4fCwCtgZigRZRLqSoDP3PPoDvXr8jrvVuGNiCTs3bvhocfhmuvdeNFPPWUPWoy+UYgdxSlfN4n4Mosvg1OOLnX/rh4pq91+XHwxTYUZb6RlATjxrkkceAANPOexFqSMPmI30ThNbQrqaoP51A8ud4zVzemRgXrBTZfWLkS+veHmTPh/PNdB35NrA6HyX/8JgpVTbRhT02+tWABrFgBH34IffvaXYTJt9JNFCJSUFUTgCgRmQh8DRxOnq+q3+VAfLnG5tgjoQ7B5ITJk92AQrfc4l5XXQXly4c6KmNCyt8dxTxcwXV5IJaTx8hWIM8niqQkZe2/B0lIVG7/yPWqXrKIVYHMk6KjYfBg+PZbaNcObr7ZNZizJGGM30QhAKp6ew7Fkuv8d9Ymnv/5RG8l59WpQI/WVhUyT0lIgLffdrWYEhLghRfcyHPWqtqYFP4SRcVUVWNPoqqvByGeXCU5SYy5uTURBQrQrlZ567Ijr1m40N1JXH65Sxi1a4c6ImNyHX+JIgIoiXdnkd8s2bYPgJoVinN508qhDcZkr/374Y8/oEcPaN8e/v7bjTxndxHGpMlfotipqsNyLJJcZPzcLbzg3U08fkWjEEdjso0qfPWVu4OIjYXNm6FKFVcmYYxJl7/6fvn269WKHfsBGHJxfetGPK/YsAG6dnW9ulatCrNnuyRhjMmQvzuKbhmtLCIlVfVQNsYTcq/9toYpK/+hVNGC3H9xvVCHY7LDwYPQpo1rZT1qFNxzD0REhDoqY8KGv0QxTkSigB+Bhap6GEBEagNdgF7A+8A3wQ4y2Nb/e4jXp6whPlGZuS6G0kUL0af9WaEOy5yupUuheXMoVco1mjvnHHc3YYzJlHQfPanqRcAfuC7FV4jIfhGJBcYDZwK3qWrYJwmAp35YxuRlu9gSe5hakSV58spGdjcRzmJi4LbboEUL14AO4LrrLEkYk0UZdeExGZic1Y2LyOXASFwNqg9UdXg6y50NzAV653TyiTueyNyNewD4+b6OFIqwbhrCVlIS/Pe/8MgjrhvwJ56Azp1DHZUxYS+QoVC/EZErRCRTV1CvQ8G3ga5AY+BGEWmcznIvA79mZvvZZf5mlyR6tqlmSSLcXXcd3H236+E1Kso1nituHTgac7oCuTKOAfoA60RkuIg0DHDb7YD1qrpRVY8DXwDXpLHcvbhuy/8NcLvZKkkVgBvb2zDgYenwYdeiGuDGG12X4NOmQeNTvpMYY7Iow0Shqr+rah9cv0+bgSkiMltEbheRQn5WrQps8/kc7U1LISJVgWtxyShdItJPRBaIyIKYmHw3ZpJJz08/uYTwzjvuc69ermzCGs4Zk60CetYiIhWAvsBdwGJcuUNrYIq/1dKYpqk+vwk8qqqJ/vavqmNVta2qtq1YsWIgIZu8bNs216q6WzdXo6lNm1BHZEyeFshQqN8BDYFPgatVdac360sRWeBn1Wigus/nasCOVMu0Bb4Q9w0wErhCRBJU9YfAwjf5zvjxbjChpCQYPhyGDIHChUMdlTF5WiB9Zn/g1X5KISJFVPWYqrb1s958oJ6I1AK2AzcAN/kuoKq1fLY5DphkScKkSdU9UqpWzdVkeustqFUrw9WMMacvkEdPz6cxbU5GK3mDHg3C1WZaBXylqitEpL+I9M9cmCbf2rcPBgxwY1aDSxKTJlmSMCYH+Rvh7kxc4XMxEWnFiTKH0kBAdQ7TaoehqmkWXKtq30C2mZ2+WxTN0z+uAPJxx1a5lSpMmAAPPOAa0A0ZcuKuwhiTo/w9eroMV4BdDfAde+Ig8EQQY8oxK3Yc4FhCIoO61KVxldKhDsck27QJ+vWD33933X//8gu0ahXqqIzJt9JNFKr6MfCxiFynqt/mYEw5qkjBCB66rEGowzC+4uNdP01vvw3/+Y914GdMiPl79HSzqo4HaqY10l24j3C3efdhPvxrE0UKWmvsXOGPP+Dnn+H116F+fdiyBYoWDXVUxhj8F2aX8H6WBEql8QpbczbE8sg3SwE4v25kiKPJ5/75B26+GS6+GCZOdAMKgSUJY3IRf4+e3vPevqOqeaY59KSlO3hjylo2xx6hSZXSvHNz61CHlD8lJcH778Njj7luOP7v/+Dxx6FYsVBHZoxJJZB2FLNFZBPwJfCdqu4NckxBszR6H4M+XwxAtxZVGHWjFZCGzP798NRT0LIlvPsuNAy0CzFjTE4LpK+nesBTQBNgoYhMEpGbgx5ZEPzfD8sBeObqxpYkQuHQIVcGkZgI5crB33/Dn39akjAmlwuoJFdV56nqA7geYfcAHwc1qiBY989B9sXF065meW7vYI21ctyPP7oO/B58EKZPd9Nq17Z2EcaEgUDGoygtIreJyC/AbGAnLmGEjf1x8Vzyxgy2xB6hSlkrJM1RW7bANddA9+5QtizMmgUXXhjqqIwxmRBIGcUS4AdgmKpm2HVHbvTWH+sAuKdzHfp3rhPiaPIRVbj+eli5El55BQYPhkL+eqY3xuRGgSSK2qqaunvwXC855FnrY/ngr00A3HdRPYoWssZbQTd3LjRp4roAHzsWypeHs84KdVTGmCzy1+DuTVUdDEwUkVMShap2C2Zgp6v32LnM27Qn5fPwHs0sSQTbnj2uiuvYsfD00/Dss9b1hjF5gL87ik+9nyNyIpDstuHfQzSvVoYLG55BhRKF6X129YxXMlmj6saJePBBlywefPBEb6/GmLDnr8HdQu9tS1Ud6TtPRO4HpgczsOzQvFoZBl9cP9Rh5H1PPOEGETrnHJgyBVq0CHVExphsFEj12NvSmNY3m+PIVtv2HCH28PFQh5G3HT0Ku3e797ff7hrNzZplScKYPMhfGcWNuBHpaonIRJ9ZpYDYYAd2Ol7+32oA6lQsGeJI8qgpU+Cee6BpU/j+e9eJX327czMmr/JXRpHcZiISeM1n+kFgaTCDOh3xiUlMWuqG9e7V1solstWuXW4goQkToF49GDQo1BEZY3KAvzKKLcAW4NycC+f0rf3nIAAd60VSokggtX9NQKZOhWuvhbg4GDoUHn3Ueng1Jp/w9+jpL1U9X0QOAr7VYwVQVc2VQ8JF740D4JZzrN5+toiPd43kmjeHSy6BF16wx0zG5DP+7ijO936G1dgTr/66BoCKpYqEOJIwd/CgawsxZ44rpK5QAb7+OtRRGWNCIJC+nuqISBHvfWcRuU9EygY9siw4lpDI+n8PAdCqRrkQRxOmVOG776BRIxg50jWYO3Ys1FEZY0IokOqx3wKJIlIX+BCoBXwe1KiyaI9XJXZQl7ohjiRM7d4NV18N110HkZEwe7ar9lq8eKgjM8aEUCCJIklVE4BrgTdVdQhQObhhZc7R+ER2HzpGt9GzAHvslGWlSrmhSV9/HRYscA3ojDH5XiDVguK9NhW3AVd703JNF6BJSUrHV6YSc9A9Hrn5nBpc36ZaiKMKI3/95Qqov/4aSpZ0gwkVCGiYEmNMPhHIFeF2XBXZF1R1k4jUAsYHN6zATVn1DzEHj3FRwzN4vVcLhnVratViAxEbC3fdBR07um7AN2500y1JGGNSyfCKqqorgft8Pm8ChgczqMzYtucIAI91bUi9SmFVQSs0VOHjj+Ghh2DfPtd53zPPQIkSoY7MGJNLZZgoRKQDMBQ4y1s+uR1F7eCGljlnlrHGXwH75BNo0ADGjIFmzUIdjTEmlwvkGc2HwBBgIZAY3HBMUMTFud5d774bqlWDb7+FMmXsMZMxJiCBJIr9qvpL0CMxwfHrr64Dv40b4YwzYOBAKGdtTIwxgQskUUwVkVeB74CUllequihoUZnTt2MHDBkCX33lHjP9+Sd06RLqqIwxYSiQRNHe+9nWZ5oCF2Z/OCbbPP88/PgjDBsGjzwCRaxtiTEmawKp9WRfQ8PFwoUnOvB77jnXJXhda6VujDk9gfT1VElEPhSRX7zPjUXkzkA2LiKXi8gaEVkvIo+lMb+PiCz1XrNFJNPDo22IOZzZVfKeAwfgvvugXTs3LCm4TvwsSRhjskEg1V7GAb8CVbzPa4HBGa0kIhHA20BXoDFwo4g0TrXYJuACVW0OPAeMDShqH78sd4MUFS6YD2vwqLoW1Q0bwujRMGAAjM81bSGNMXlEIFfXSFX9CkgC8Pp9CqSabDtgvapuVNXjwBfANb4LqOpsVd3rfZwLZLrvjRKFC9KxXiRFCkZkdtXw9/nn0KsXnHmm63pj9GgoWzbUURlj8phACrMPi0gFvMGLROQcYH8A61UFtvl8juZEwXha7gTSrIYrIv2AfgA1atRIma6qHDqWkL86ATx+3FV1bdgQrr/etZHo2xcKWrclxpjgCOTq8gAwEagjIrOAisD1AawnaUzTNKYhIl1wieL8tOar6li8x1Jt27ZN2cbj3y1jf1w8hfJLw7EZM6B/fzh0CNaudUOR3nVXqKMyxuRxgdR6WiQiFwANcBf/NaoaH8C2o4HqPp+rATtSLyQizYEPgK6qGhtQ1J7t+9ywpwPz+vgTu3e7PpnGjYOaNV3XGzZetTEmh/gbM/tsYJuq7lLVBBFpA1wHbBGRoaq6J4Ntzwfqeb3NbgduAG5KtY8auIZ8t6jq2swE3ueDuczftJfWNcpSo0IeHlhn40Y4+2xXs+mxx+D//s8GEjLG5Ch/z2zeA44DiEgnXI+xn+DKJzKsneQVeg/C1ZhaBXylqitEpL+I9PcWexqoALwjIlEisiCQoJdG72PW+ljqVSpJ/wvqBLJK+DlwwP2sVQtuvx0WL4aXXrIkYYzJcaKaZrEBIrJEVVt4798GYlR1qPc5SlVb5lSQvlq2bqP7Lh0GwCd3tKNT/YqhCCN4jhxxjeXGjoUlS1wnfsYYc5pEZKGqts14yVP5u6OIEJHkR1MXAX/6zAtZFZujx13N3JoVitOhbmSowgiOn3+GJk1cT6/XXAPFioU6ImOM8XvBnwBMF5HdQBwwE0BE6hJY9digeq1XCyIKpFWxKgwlJMCNN8I330CjRjB9OnTqFOqojDEG8JMoVPUFEfkDqAz8pieeURUA7s2J4PI8VRBxbSAqVYIXX4QHH4TChUMdmTHGpPD7CElV56YxLVO1k0w65s93Y0OMGQOtW7tW1cYYkwuFXUu1+KSkUIdwevbvh0GDoH17iI6G2Ew1HTHGmBwXdoli72HX1q900UIhjiQLkjvwe/ddlyxWr4ZLLgl1VMYY41fYdRCkKJElC1OvUqlQh5J5q1ZB1arw00/QNku11IwxJseF3R3FkeOJ4XM3ceyYG2nup5/c58cfd728WpIwxoSRsEsUAJc2OTPUIWRs6lRo0cJ1ufHHH25aoUIQkQ+7QzfGhLWwTBR1KpYIdQjp+/dfuO02uPBCiI+HX36BN98MdVTGGJNlYZkocrXffoMJE+DJJ2H5crj88lBHZIwxpyXsCrNzpWXLYM0aN5BQnz5w3nlQu3aoozLGmGxhdxSn4/BheOQRaNXK/YyPdy2tLUkYY/IQu6PIqp9+cm0htm6FO++El192hdX5XHx8PNHR0Rw9ejTUoRiTLxUtWpRq1apRKBuvR5YosmL5cujWzfX0OnMmnJ/mCK75UnR0NKVKlaJmzZqI5JFOG40JE6pKbGws0dHR1KpVK9u2a4+eApWQANOmufdNm8KkSW4wIUsSJzl69CgVKlSwJGFMCIgIFSpUyPY7eksUgUhuJHfRRbBunZt25ZX2qCkdliSMCZ1g/P9ZovBn714YMADOPRd273Z9NdWtG+qojDEmR1miSM+xY64209ixMHiw66epRw9Xq8nket9//z0iwurVq9NdpnPnzixY4H+Y9po1a7J79+7sDg+AqKgoJk+enO78efPm0alTJxo0aEDDhg256667OHLkCOPGjWPQoEHZFscVV1zBvn37ABg1ahSNGjWiT58+TJw4keHDh2d5u5s3b6ZYsWK0bNmSxo0bc+uttxIfH58y/6+//qJdu3Y0bNiQhg0bMnbs2JPW/+STT2jatClNmjShcePGjBgxIs39vPnmm3zyySdZjjPYNm3aRPv27alXrx69e/fm+PHjaS63detWLr30Uho1akTjxo3ZvHmz3/UnTZrEM888kzMHoaph9Sp8Zl39av5WDZro6BPvP/pIddGi4O0rD1q5cmWoQ1BV1Z49e+r555+vzzzzTLrLXHDBBTp//ny/2znrrLM0JiYmm6NzPvroIx04cGCa83bt2qU1atTQ2bNnq6pqUlKSfv3117pr1y6/652uBg0a6MaNG7O0bnx8/EmfN23apE2aNFFV1YSEBO3SpYuOHz9eVVV37typ1atX14ULF6qqakxMjLZu3VonTZqkqqqTJ0/WVq1a6fbt21VVNS4uTseOHZvmPps1a3bKvjMTZ7D17NlTJ0yYoKqq//nPf/Sdd95Jc7kLLrhAf/vtN1VVPXjwoB4+fNjv+klJSdqyZcuU5Xyl9X8ILNAsXnet1lOyo0ddFdcXX4SvvnJjVvftG+qowtqzP61g5Y4D2brNxlVK88zVTfwuc+jQIWbNmsXUqVPp1q0bQ4cOBSAuLo7bb7+dlStX0qhRI+Li4lLWGTBgAPPnzycuLo7rr7+eZ599NmXeq6++ytSpUwH4/PPPqVu3Llu2bOGOO+4gJiaGihUr8tFHH1GjRo10p3/99dc8++yzREREUKZMGX7//Xeefvpp4uLi+Ouvv3j88cfp3bt3yj7ffvttbrvtNs4991zAPXe+/vrrTznWn376ieeff57jx49ToUIFPvvsMypVqsT06dO5//77U9adMWMGhw4donfv3hw4cICEhATeffddOnbsSM2aNVmwYAFPPfUUGzdupFu3btxxxx2UK1eOBQsWMHr0aGJiYujfvz9bt24F3Lf4Dh06MHToUHbs2MHmzZuJjIzk888/T/N3EhERQbt27di+fXvK8fXt25fWrVsDEBkZySuvvMLQoUO58soreemllxgxYgRVqlQBXJXPu++++5Tt/vnnn7Ru3ZqCBd2l7P3332fs2LEcP36cunXr8umnn1K8eHH69u1L+fLlWbx4Ma1bt+aee+5h4MCBxMTEULx4cd5//30aNmyY7vnMKlXlzz//TDkvt912G0OHDmXAgAEnLbdy5UoSEhK4xBt2oGTJkhmuLyJ07tyZSZMm0atXryzHGAh79ASu077mzWHoULjuOjeokAlbP/zwA5dffjn169enfPnyLFq0CIB3332X4sWLs3TpUp588kkWLlyYss4LL7zAggULWLp0KdOnT2fp0qUp80qXLs28efMYNGgQgwcPBmDQoEHceuutLF26lD59+nDffff5nT5s2DB+/fVXlixZwsSJEylcuDDDhg2jd+/eREVFnZQkAJYvX06bNm0yPNbzzz+fuXPnsnjxYm644QZeeeUVAEaMGMHbb79NVFQUM2fOpFixYnz++edcdtllREVFsWTJElq2bHnStsaMGUOVKlWYOnUqQ4YMOWne/fffz5AhQ5g/fz7ffvstd911V8q8hQsX8uOPP6abJMDVhvv777+53OvSZsWKFaccX9u2bVmxYkWmjn/WrFknLdejRw/mz5/PkiVLaNSoER9++GHKvLVr1/L777/z2muv0a9fP9566y0WLlzIiBEjuOeee/yeT19r1qyhZcuWab6SH+Eli42NpWzZsimJrFq1ainJ0tfatWspW7YsPXr0oFWrVjz88MMkJiZmuH7btm2ZOXNmhufpdNkdxeDBMHKkK6T+7TcbSCgbZfTNP1gmTJiQckG/4YYbmDBhAq1bt2bGjBkpF+7mzZvTvHnzlHW++uorxo4dS0JCAjt37mTlypUp82+88caUn8kX0Dlz5vDdd98BcMstt/DII4/4nd6hQwf69u1Lr1696NGjR7Yda3R0NL1792bnzp0cP348pe58hw4deOCBB+jTpw89evSgWrVqnH322dxxxx3Ex8fTvXv3UxKFP7///jsrV65M+XzgwAEOHjwIQLdu3ShWrFia623YsIGWLVuybt06rr/++pRzqqpp1s7JbI2dnTt30qhRo5TPy5cv56mnnmLfvn0cOnSIyy67LGVez549iYiI4NChQ8yePZuePXumzDt27BiQ/vn01aBBA6KiogKKzz3xOVlax5iQkMDMmTNZvHgxNWrUoHfv3owbN45u3br5Xf+MM85gx44dAcVyOvLnHUVSEiQmuvft2sHTT7v+mixJhL3Y2Fj+/PNP7rrrLmrWrMmrr77Kl19+mfIPm9Y/6aZNmxgxYgR//PEHS5cu5corrzypHrrvOuldyDKaPmbMGJ5//nm2bdtGy5Ytic1gCNwmTZqcdMeTnnvvvZdBgwaxbNky3nvvvZS4H3vsMT744APi4uI455xzWL16NZ06dWLGjBlUrVqVW265JVMFwElJScyZM4eoqCiioqLYvn07pUq5wcNKlEi/N+c6deoQFRXF+vXrmTt3LhMnTkw5vtQVCRYuXEjjxo0zdfzFihU76XfVt29fRo8ezbJly3jmmWdOmpccZ1JSEmXLlk05lqioKFatWgWkfz59ZeaOIjIykn379pGQkAC4RJT8OM1XtWrVaNWqFbVr16ZgwYJ0796dRYsWZbj+0aNH003S2Sn/JYolS1ynfW+/7T7fdBM8+ywULRrauEy2+Oabb7j11lvZsmULmzdvZtu2bdSqVYu//vqLTp068dlnnwHum2fy46UDBw5QokQJypQpwz///MMvv/xy0ja//PLLlJ/JZQbnnXceX3zxBQCfffYZ53sNL9ObvmHDBtq3b8+wYcOIjIxk27ZtlCpVKuVbeWqDBg3i448/5u+//06ZNn78eHbt2nXScvv376dq1aoAfPzxxynTN2zYQLNmzXj00Udp27Ytq1evZsuWLZxxxhncfffd3HnnnSmP5AJx6aWXMnr06JTPgX6jTla5cmWGDx/OSy+9BMDAgQMZN25cynZiY2N59NFHU+7AHn/8cR555JGU4z127BijRo06ZbuNGjVi/fr1KZ8PHjxI5cqViY+PT/ldp1a6dGlq1arF119/Dbhv/UuWLAHSP5++ku8o0nqVLVv2pGVFhC5duvDNN9+kbPOaa645ZZtnn302e/fuJSYmBnBlL40bN85w/bVr19K0adM048xO+SdRHDoEDz4IbdrAxo1wZhgMfmQybcKECVx77bUnTbvuuuv4/PPPGTBgAIcOHaJ58+a88sortGvXDoAWLVrQqlUrmjRpwh133EGHDh1OWv/YsWO0b9+ekSNH8sYbbwCuGulHH31E8+bN+fTTTxk5cqTf6Q8//DDNmjWjadOmdOrUiRYtWtClSxdWrlxJy5YtU5JRskqVKvHFF1/w0EMP0aBBAxo1asTMmTMpXbr0ScsNHTqUnj170rFjRyIjI1Omv/nmmzRt2pQWLVpQrFgxunbtyrRp02jZsiWtWrXi22+/TSnsDsSoUaNYsGABzZs3p3HjxowZMybgdZN1796dI0eOMHPmTCpXrsz48eO5++67adiwIeeddx533HEHV199NeCq7A4cOJCLL76YJk2a0KZNm5Rv1b66du3KjBkzUj4/99xztG/fnksuuYSGDRumG8tnn33Ghx9+SIsWLWjSpAk//vgjkP75PB0vv/wyr7/+OnXr1iU2NpY777wTgAULFqSU9URERDBixAguuugimjVrhqqmFN6ntz7A1KlTufLKK7MlTr+yWl0qVK8sVY+dMkW1WjVVUO3XT3XPnsytbwKWW6rHmvyje/fuunbt2lCHkeN27dqlF154YZrzsrt6bFjeUTSqXDrjhXwVLgzly8OsWfDee1CuXHACM8bkuOHDh7Nz585Qh5Hjtm7dymuvvZYj+wq7Wk8FRGhatYz/heLj3fCj+/fD889Dp06uA78CYZkXjTF+NGjQgAYNGoQ6jBx39tln59i+8t6Vc/ZsVw7xyCOu242kJDfdkkSO0TSqBBpjckYw/v/yztVzzx7o1w86dIB9++CHH+Dbby1B5LCiRYsSGxtrycKYEFB141EUzeZanGH36CldsbHw+efw0EPwzDPgNYE3OatatWpER0enVPMzxuSs5BHuslN4J4o1a+DLL12DuXr1YMsWqFAh1FHla4UKFcrWkbWMMaEX1OcyInK5iKwRkfUi8lga80VERnnzl4pI64A2HBfnkkPz5vDGG7Btm5tuScIYY7Jd0BKFiEQAbwNdgcbAjSLSONViXYF63qsf8G5G2y157DA0awbPPQc9e8Lq1VC9ejZHb4wxJlkw7yjaAetVdaOqHge+AFK3Xb8G+MRrDzIXKCsilf1ttOq+f1wB9e+/w/jxcBpdABtjjMlYMMsoqgLbfD5HA6n7705rmarASa1nRKQf7o4D4JisW7eciy/O3mjDUyQQnOHXwo+dixPsXJxg5+KELDc2CWaiSKs7zdR1JgNZBlUdC4wFEJEFqtr29MMLf3YuTrBzcYKdixPsXJwgIv7H/fUjmI+eogHfwoNqQOqO0wNZxhhjTAgFM1HMB+qJSC0RKQzcAExMtcxE4Fav9tM5wH5VzX+dthhjTC4WtEdPqpogIoOAX4EI4L+qukJE+nvzxwCTgSuA9cAR4PYANj02SCGHIzsXJ9i5OMHOxQl2Lk7I8rkQ62rBGGOMP9YRkjHGGL8sURhjjPEr1yaKoHX/EYYCOBd9vHOwVERmi0iLUMSZEzI6Fz7LnS0iiSJyfU7Gl5MCORci0llEokRkhYhMz+kYc0oA/yNlROQnEVninYtAykPDjoj8V0T+FZHl6czP2nUzq0PjBfOFK/zeANQGCgNLgMaplrkC+AXXFuMc4O9Qxx3Cc3EeUM573zU/nwuf5f7EVZa4PtRxh/DvoiywEqjhfT4j1HGH8Fw8Abzsva8I7AEKhzr2IJyLTkBrYHk687N03cytdxRB6f4jTGV4LlR1tqru9T7OxbVHyYsC+bsAuBf4Fvg3J4PLYYGci5uA71R1K4Cq5tXzEci5UKCUiAhQEpcoEnI2zOBT1Rm4Y0tPlq6buTVRpNe1R2aXyQsye5x34r4x5EUZngsRqQpcC4zJwbhCIZC/i/pAORGZJiILReTWHIsuZwVyLkYDjXANepcB96tqUs6El6tk6bqZW8ejyLbuP/KAgI9TRLrgEsX5QY0odAI5F28Cj6pqovvymGcFci4KAm2Ai4BiwBwRmauqa4MdXA4L5FxcBkQBFwJ1gCkiMlNVDwQ5ttwmS9fN3JoorPuPEwI6ThFpDnwAdFXV2ByKLacFci7aAl94SSISuEJEElT1hxyJMOcE+j+yW1UPA4dFZAbQAshriSKQc3E7MFzdg/r1IrIJaAjMy5kQc40sXTdz66Mn6/7jhAzPhYjUAL4DbsmD3xZ9ZXguVLWWqtZU1ZrAN8A9eTBJQGD/Iz8CHUWkoIgUx/XevCqH48wJgZyLrbg7K0SkEq4n1Y05GmXukKXrZq68o9Dgdf8RdgI8F08DFYB3vG/SCZoHe8wM8FzkC4GcC1VdJSL/A5YCScAHqppmtclwFuDfxXPAOBFZhnv88qiq5rnux0VkAtAZiBSRaOAZoBCc3nXTuvAwxhjjV2599GSMMSaXsERhjDHGL0sUxhhj/LJEYYwxxi9LFMYYY/yyRGFOklHvkz7LPen1wrnU6520fTbHMVlEynrv7xORVSLymYh089drrLf8bO9nTRG5KcD9dReRp733Q0Vku3dcUSIy3M96Q0XkoYAPLO1t1BSROG9fK0VkjIhk6n9TRNqKyCjvfWcROc9nXv/s6L4j1XlZKSI3BrDOYK8NR0bLfSEi9U43RhMcVj3WnEREOgGHcB2HNU1nmXOB14HOqnpMRCJxPXEGpWW8iKzGtTjflMn1OgMPqepVASw7G+imqrtFZChwSFVHBLBewMv62UZNYJKqNhWRgrieb99U1e+yuL3Tjimj7XoX9YVABVWN97POZqBtRm0WROQC4GZVvTsbQzbZxO4ozEkC6H0SoDKua4hj3jq7k5OEiGwWkZdFZJ73qutNrygi34rIfO/VwZteUkQ+EpFl3t3JdT7biRSRMbjuoyeKyBAR6Ssio71lKonI9+LGGFiS/C1aRA55cQ7HtUyO8tadKSItkw9CRGaJSHMRqQ8c83cxE5G7vbiXeMdxyrdk785npXccX3jTSnh3afNFZLGIpNXbre/5TwBmA3VF5CwR+cPb3h/iWuAjIj1FZLkXywxvWmcRmeQlnf7AEO+4Oybf9YhIIxFJ6bLCu5NZ6r1vIyLTxXUe+Ktk0KOoqq7DNdgq563/rogsEHeX+Wzy+QCqAFNFZKo37VIRmSMii0TkaxEp6W1yJnCxlyhNbpPT/aXbK/e/gJqk05+9N78kroO1tcA7wAU+8zYDT3rvb8V9Uwb4HDjfe18DWOW9fxn37Tl5/XI+24lM431fYLT3/ktgsPc+AijjvT/k/eycvH/v823J+8L1rLrAe3878JrPckOB7d4xRuE6lKvgM/954F6fZR/y3u8Ainjvy3o/X8R9UwY3PsRaoER65xsojuuSoivwE3CbN/0O4Afv/TKgaqr9pByrb0xpxBgF1PbePwo8hWu5Oxuo6E3vjWvdnPr37rud1sBMn3nlfX4P04DmafzuIoEZycfv7f9pn21MAdqE+u/fXqe+7I7CZJqqHsL1StoPiAG+FJG+PotM8Pl5rvf+YmC0iETh+pspLSKlvOlv+2x7L4G7EHjXWy9RVfdnsPzXwFUiUgh34R3nTa/sHYevN1S1pff6FWjq3ZEsA/oATdLY/lLgMxG5mRNjHVwKPOYd9zSgKC5RplbHW2YW8LOq/oI7d5978z/lRK/As3DdUdyNuzBnxldAL+99b1yybQA0xfWoGoVLHumNaTJERNYAf+MSR7JeIrIIWIw7N43TWPccb/osbz+3AWf5zP8Xdwdichm7zTMZEpHquG+3AGPU9SOUiLvwTfMunrdx4sLrW/CV/L4AcK6qxqXatpBD3cOr6hERmYIbvKUXrqdZgDigTAarjwO6q+oSLyl2TmOZK3EjjHUD/k9EmuD6FbpOVddksP0Nqtoyo0PwjqO/uMoDVwJRvo/TAvAl8LWIfOc2petEpBmwQlXPzWBdcAl0hIj0AD4RkTq4RPsQcLaq7hWRcbiEmJoAU1Q1vULworjfhcll7I7CZEhVt/l8ux4jIg3k5BoqLYEtPp97+/yc473/DRiUvIDPxS319HKZCO0PYIC3XoSIlE41/yBQKtW0D4BRwHxVTS6LWQXUzWBfpYCd3t1In9QzxdVSqq6qU4FHcI+ZSuI6qrvXS4iISKvADg1wj4Nu8N73Af7ytlFHVf9W1aeB3ZzcbTSkfdwAqOoGIBH4P1zSAFgDVBRXSQERKeQluXSpK2hfgPuCUBo4DOwX1zNr13RimQt0kBPlVsW98qFk9YEV/vZrQsMShTmJuN4n5wANRCRaRO5MY7GSwMfJBbe4xwlDfeYXEZG/gfuBId60+4C2XsHsSlyBK7jn/eWSC2eBLpkI936gi3dHs5BTHwctBRK8Qt8hAKq6EDgAfOSz3AygVfLFPB3/h3vcMgVYncb8CGC8F8ti3DfvfbheSwsBS8VVOX4uE8d3H3C7d45v8Y4X4FVxhf/LvdiXpFrvJ+Da5MLsNLb7JXAz7jEU6oYPvR542fsdROHGYc/IMOABXJnJYtxF/r+4R2PJxgK/iMhUVY3BlTFN8I5pLm5MiOSuv+M0bw4VEPaseqzJVhJgdchQEZEquEdmDdVnKEwRGQn8pKq/hyq2/MxL5AdU9cNQx2JOZXcUJt8Q1+jsb1ytrNTjJb+Iq3FkQmMf8HGogzBpszsKY4wxftkdhTHGGL8sURhjjPHLEoUxxhi/LFEYY4zxyxKFMcYYv/4fkhydN5K7sRcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## Plot ROC AUC Curve\n",
    "from sklearn.metrics import roc_auc_score,roc_curve\n",
    "plt.figure()\n",
    "\n",
    "# Add the models to the list that you want to view on the ROC plot\n",
    "auc_models = [\n",
    "{\n",
    "    'label': 'Adaboost Classifier',\n",
    "    'model':AdaBoostClassifier(n_estimators=80, algorithm='SAMME'),\n",
    "    'auc':  0.6049\n",
    "},\n",
    "    \n",
    "]\n",
    "# create loop through all model\n",
    "for algo in auc_models:\n",
    "    model = algo['model'] # select the model\n",
    "    model.fit(X_train, y_train) # train the model\n",
    "# Compute False postive rate, and True positive rate\n",
    "    fpr, tpr, thresholds = roc_curve(y_test, model.predict_proba(X_test)[:,1])\n",
    "# Calculate Area under the curve to display on the plot\n",
    "    plt.plot(fpr, tpr, label='%s ROC (area = %0.2f)' % (algo['label'], algo['auc']))\n",
    "# Custom settings for the plot \n",
    "plt.plot([0, 1], [0, 1],'r--')\n",
    "plt.xlim([0.0, 1.0])\n",
    "plt.ylim([0.0, 1.05])\n",
    "plt.xlabel('1-Specificity(False Positive Rate)')\n",
    "plt.ylabel('Sensitivity(True Positive Rate)')\n",
    "plt.title('Receiver Operating Characteristic')\n",
    "plt.legend(loc=\"lower right\")\n",
    "plt.savefig(\"auc.png\")\n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
